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Abstract 

We calculate Bayes factors to quantify how the feasibility of the constrained 
minimal supersymmetric standard model (CMSSM) has changed in the light of a 
series of observations. This is done in the Bayesian spirit where probability reflects 
a degree of belief in a proposition and Bayes' theorem tells us how to update 
it after acquiring new information. Our experimental baseline is the approximate 
knowledge that was available before LEP, and our comparison model is the Standard 
Model with a simple dark matter candidate. To quantify the amount by which 
experiments have altered our relative belief in the CMSSM since the baseline data 
we compute the Bayes factors that arise from learning in sequence the LEP Higgs 
constraints, the XENON100 dark matter constraints, the 2011 LHC supersymmetry 
search results, and the early 2012 LHC Higgs search results. We find that LEP and 
the LHC strongly shatter our trust in the CMSSM (with M and M 1/2 below 2 TeV), 
reducing its posterior odds by a factor of approximately two orders of magnitude. 
This reduction is largely due to substantial Occam factors induced by the LEP and 
LHC Higgs searches. 
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1 Introduction 

Supersymmetry is an attractive and robust extension of the Standard Model of 
particle physics [I] . Weak scale supersymmetry resolves various shortcomings of the 
Standard Model, and explains several of its puzzling features (2||7j. Coupled with 
high-scale unification, supersymmetry breaking radiatively induces the breakdown 
of the electroweak symmetry. It also tames the quantum corrections to the Higgs 
mass, provides viable dark matter candidates, and is able to accommodate massive 
neutrinos and explain the cosmological matter-antimatter asymmetry [8}jl2]. It is 
also an ideal framework to address cosmological inflation [l3j[l4j . 

However, to date there is no experimental data providing direct evidence for 
supersymmetry in Nature. The exclusion of supersymmetric models based on ob- 
servation proves to be just as difficult as discovery, because the large number of 
parameters in the supersymmetry breaking sector makes supersymmetry (SUSY) 
sufficiently flexible to accommodate most experimental constraints. The most pre- 
dictive supersymmetric models are the constrained ones where theoretical assump- 
tions about supersymmetry breaking are invoked, reducing the number of free pa- 
rameters typically to a few. 

The most studied SUSY theory is the constrained minimal supersymmetric stan- 
dard model (CMSSM) (l5|[l6j. Motivated by supergravity, in the CMSSM the spin-0 
and spin- 1/2 super-partners acquire common masses, Mq and M1/2? an d trilinear 
couplings, Ao, at the unification scale. The Higgs sector is parameterised by the 
ratio of the Higgs doublet vacuum expectation values (VEVs), tan/3, and the sign 
of the higgsino mass parameter, sign /1. 
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Based on experimental data, an extensive literature delineates the regions of 
the CMSSM where its parameters can most probably fall. After the early intro- 



duction of x 2 as a simple measure of parameter viability [17, 18 increasingly more 
sophisticated concepts were utilised, such as the profile likelihood and marginalised 
posterior probability and the corresponding confidence 19-22] or credible [23, 24 



regions. The effect of the LHC data on the CMSSM has typically been presented in 
this general manner both in the frequentist 25 -27] and the Bayesian [28}|3T] frame- 



work. To go beyond parameter estimation and obtain a measure of the viability of 
a model itself one has several options. The most common frequentist measure is 
the p- value, the probability of obtaining more extreme data than the observed from 
the assumed theor}^] |32, 33 1. In the Bayesian approach model selection is based on 



the Bayes factor, and requires comparison to alternative hypotheses. 1 34 39 

In the Bayesian framework the plausibility of the CMSSM can only be assessed 
when we consider it as one of a mutually exclusive and exhaustive set of hypotheses: 
CMSSM E {Hi}. The posterior probabilities of each of these hypotheses, in light 
of certain data, are given by Bayes' theorem 

Since the denominator in the right hand side is impossible to calculate, it is advan- 
tageous to compare the plausibility of the CMSSM to that of a reference model by 
forming the ratio 

Here we have selected the Standard Model as our reference hypothesis. Using eq.Q 
we can rewrite the odds in terms of likelihood ratios as 

^ , , /n„nm, nwi, \ P(data I CMSSM) P(CMSSM) 
Odds (CMSSM :SM \data) = V T , / - 1 — - 7 ^ /r ^ x = 
v 1 J P(data\SM) P(SM) 

B(data\CMSSM:SM) O dds (CMSSM :SM). (3) 

The second ratio on the right hand side is called the prior odds, and is incalcu- 
lable within the Bayesian approach. The first ratio, however, is calculable, and is 
commonly called the 'Bayes factor'. It gives the change of odds due to the newly 
acquired information. 

In this work we quantify the change of plausibility of the CMSSM (compared to 
that of the SM) by examining how new data change the CMSSM:SM odds. We do 
this by computing a series of Bayes factors associated with learning the results of 
several LHC and other experiments. These Bayes factors then quantify the 'damage' 
that each of these experiments has done to the CMSSM. To evaluate the relevant 
Bayes factors we will need to calculate marginalised likelihoods (or evidences) such 
as P(data | CMSSM). These are calculated as integrals over the CMSSM parameters 

P(data\CMSSM) = J P(data\CMSSM 1 9)P(9\CMSSM)d9. (4) 

Here the notation P(data\ CMSSM, 9) is understood as distinct from P(data\ CMSSM): 
the latter is the Bayesian probability of observing data if an unknown point in the 
general CMSSM framework is generating it (computed by the marginalisation in- 
tegral of eq.Q), while P(rfata|CMSSM, 9) admits a standard frequentist interpre- 
tation as the probability of the data assuming the CMSSM parameter space point 



x Here 'more extreme' can be defined in numerous ways. 
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9 to be generating it, i.e. as a likelihood function. While the likelihood function 
is uncontroversial, the choice of P(#| CMSSM) describing the a priori distribution 
of the parameters is somewhat subjective. We fix this prior based on naturalness 
arguments. The underlying idea is that the mechanism protecting the Higgs mass 
from quantum fluctuations has to be natural (40j . If supersymmetry is a natural 
mechanism for this then gaugino masses have to be light |4l||5l] . 

We use this naturalness argument as theoretical motivation to quantify the 
distribution of CMSSM parameters before observations, following previous studies 



24,44,52-641. We call the resulting P(6 | CMSSM) distribution the natural prior. 



To investigate the dependence of our results on this prior we also calculate Bayes 
factors using logarithmic priors. 



2 Bayesian 'updating' 

The Bayesian framework describes how to update probabilities of competing propo- 
sitions based on newly acquired information, where 'probability' is interpreted as 
measuring a 'degree of belief in competing propositions |55|. This probability is 
subjective insofar as it depends upon a subjective 'starting point', i.e. an initial set 
of prior odds, but the updating procedure is completely objective, and dependence 
on the 'starting point' diminishes as data is acquired. As eq.Q shows, the prior 
odds are updated to better reflect reality by multiplying them by Bayes factors to 
form posterior odds. These Bayes factors therefore quantify the effect of the new 
information on the odds. 

It is easy to prove that once further information is available we can consider the 
earlier posterior odds as prior and fold in the new information by just multiplying 
these odds with a new Bayes factor. To show this, we assume that there exist two 
sets of data, d\ and d 2l and we examine their effect on the prior odds. Using eq.Q 
the posterior odds containing both d\ and d 2 can be written as 

^r, , , N P ( d 2 | ^1 , CMSSM) P(CMSSM | d X ) 

Comparing the first term on the right hand side to eq.Q we identify the Bayes 
factor induced by d 2 . Making this explicit we obtain 

Orfrfs(CMSSM:SM|rfi,rf 2 ) = B(d 2 \d 1 , CMSSM : SM) P ^^,^f ^ • (6) 

Applying eq.Q again on the last term above, this transforms into 

Odds(CMSSM : SM|di, d 2 ) = B(d 2 \d u CMSSM : SM)B(d 1 |CMSSM : SM) P ^^ M ^ . 

(7) 

By induction the above holds for any set of data {d\, cfo, d n }. That is Bayes 
factors factorise and update the odds multiplicatively. 

Odds(CMSSM:SM|di,d2,».,dn)= ^JJfl(di|di_i, CMSSM :SM)^ . 

(8) 

The size of a Bayes factor induced by a certain set of data depends on what 
other data is already known and folded into the odds. This can be understood 
by considering the following example. Assume that data set d\ excludes a certain 
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portion of the CMSSM parameter space, and d 2 excludes another portion that is 
fully contained within the portion that d\ excluded (for simplicity we assume that 
d\ and d 2 have no effect on the SM). If we learn d\ first its Bayes factor updates 
the prior odds by £>(<ii|CMSSM : SM). Learning d 2 after this changes nothing so 
its induced Bayes factor must be unity B{d 2 \d\ 1 CMSSM : SM) = 1. In contrast, 
when learning d\ first and then d 2 their Bayes factors, B{d 2 \ CMSSM : SM) and 
B(d\\d 2l CMSSM : SM), both have to be less than one, while their product must 
equal B(d u d 2 |CMSSM : SM). 

Since Bayes factors do not 'commute' it is important that we define the order in 
which the data is learned. To assess the role of LEP and the LHC in constraining 
the CMSSM we deviate slightly from the historic order in which data appeared. We 
assume that the initial odds contains information from various LEP direct sparticle 
search limits, the neutralino relic abundance, muon anomalous magnetic moment, 
precision electroweak measurements and various flavour physics observables. This 
set of data forms our baseline. We then compute the Bayes factors induced by 
folding in the LEP Higgs search and XENON100 dark matter search limits, LHC 
lfb -1 direct sparticle search limits and February 2012 LHC Higgs search results. 
These Bayes factors are then an efficient summary of how much damage has been 
done to the 'believability' of the CMSSM by this new data. Here the SM acts as a 
standard against which this damage is measured. 



3 Computing CMSSM vs SM Bayes factors 

The marginalised likelihoods, or evidences, which appear in the Bayes factor of 
eq.([5]) contain a subtle difference from the general form described in eq. @, this 
being that they are conditional on data d\\ 

P(d 2 \d u CMSSM) = Jd6P(d 2 \d u CMSSM, O)P(0\d 1 , CMSSM). (9) 

If d\ and d 2 are statistically independent then the conditioning on d\ drops out 
of the likelihood function, but it remains in the prior function P(6\d\, CMSSM). 
This prior may thus be called 'informative' because it incorporates information 
from the likelihood P{d\ | CMSSM, #), which has been folded in to the initial prior 
P(#| CMSSM), in general resulting in an extremely complicated distribution which 
makes the integral difficult to evaluate. 

Fortunately, there exists an alternative to directly evaluating the integral. From 
the definition of conditional probability we may write 

P(d\d CMSSTVn - n^ilCMSSM) 

P(d 2 \d u CMSSM) - m|CMSSM) , (10) 

where the numerator and denominator may be referred to as "global" evidences, 
since they are computed by integrating the global likelihood function over the pa- 
rameter space, with the parameter space measure defined by the initial prior dis- 
tribution for the model parameters, as is done in more conventional model compar- 



isons |56f|60| . We discuss the numerical details of the global evidence evaluation 
and priors in section |1J and the details of the global likelihood function in section 

El 

Bayes factors are only defined for a pair of hypotheses which are being compared, 
however it is useful to break them up into pieces which tell us something about what 
is happening in each hypothesis individually, so that we may more easily speculate 
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about what effect variations in one hypothesis or the other might have. While the 
evidences themselves can be examined it can be more illuminating to break them up 
further, into a contribution from the maximum of the likelihood function of the new 
data, and an Occam factor. The latter is defined only through its relationship to 
the evidence; it is what remains when the maximum value of the likelihood function 
is divided out: 

m\dl,H i)= p{MHj y (ID 
Here P(d 2 |di,i?i) is the evidence associated with learning g?2 when d\ is already 



known, as computed in eq.(M and eq.(10), and P(d 2 |iJi,0) is the maximum value 



of the likelihood function for d2 that is achieved in the model Hi (and 9 is the 
parameter space point in Hi which achieves this maximum). P(d2\Hi,9), coming 
as it does from the likelihood, does not depend on the prioij^J this dependence is 
entirely captured by the Occam factor. P(d2\Hi,9) also has no dependence on d\, 
with this dependence again contained in the Occam factor. 

These two components of the evidence give us different information about the 
model. A Bayes factor is a ratio of evidences, so by separating evidences in this 
manner we will obtain a product of ratios, one of which is a standard frequent ist 
maximum likelihood ratio (considering just the data ^2), and the other of which 
is a ratio of Occam factors. The maximum likelihood ratio tells us which model 
has the better fitting point with respect to o?2, but ignores all other aspects of the 
model and all other data. Complementing this the Occam factor tells us something 
about the relative volume of previously viable parameter space which is compatible 
with the new data ^2 in each model, where the measure of volume is defined by 
the informative prior P{9\d\ 1 Hi) 1 which has resulted from a previous Bayesian 
'update' and so "knows" about previous data d\. The Occam factor can be roughly 
interpreted as the amount by which the new data ^2 collapses the parameter space 
when it arrive^J and its logarithm as a measure of the information gained about 
the model parameters [6l] 

The impact of Occam factors on the model comparison can be seen by explicitly 
writing out the Bayes factors in terms of them: 

= P(rf 2 [rfi, CMSSM) = P(d 2 \ CMSSM, 9) 0{d 2 \d u CMSSM) 
i2|lj ~ P(d 2 |di,SM) " P(d 2 |SM,0) tf(d 2 |di,SM) ' [ } 

where <j) are the SM parameters (and <p the analogue of 9). Schematically 

t-\ T ^CMSSM / 1Q x 

B = LR x — — , (13) 

<^SM 

where LR denotes the maximum likelihood ratio and the rest of the abbreviated 
terms correspond directly to their partners in the more formal expression. We 
thus see two competing factors: a model is favoured if it achieves a high likelihood 
value for the new data somewhere in its parameter space, but disfavoured if such 
goodness-of-flt is not very compatible with our prior knowledge and data, or if it 



2 Strictly, some prior dependence remains due to the choice of parameter values considered possible by 
the prior, most often arising from the choice of scan range, however this is the same kind of dependence 
that exists in a frequent ist analysis. As well as this there exists the possibility that d\ strictly forbids 
certain values of 0, and these too should be excluded from the computation of P(d 2 \Hi,6). 

3 The full volume of the parameter space, Vtotai? is defined by the informative prior. If the likelihood 
function for the new data was constant in a region V and zero outside of it, then the fraction / = V/Vtotai 
would be the Occam factor. 



6 



only achieves a good fit in a small region compatible with the informed prior. These 
effects are also relative; i.e. no objectively "good" likelihood value is needed, just 
one which is better than achieved in alternate models, and likewise for the volume 
effects. 

Because the best fit point is only with respect to the new data it could be very 
different to the best fit point of the global likelihood function, and so may not appear 
to be a useful object to frequentist thinkers. However, in the Bayesian framework 
it is acknowledged that not all data relevant to inference can be expressed in the 
likelihood function, that is, the prior may contain real information. In our case the 
prior for each iteration (except the first) contains very concrete information; that 
coming from the rest of the likelihood. The best fit point with respect to the new 
data thus is indeed not so useful on its own (although it tells us something about 
the maximum goodness of fit possible in the model for that data), but extracting 
it from the evidence allows one to capture tension between the new and old data 
in a different way, i.e in the Occam factor. 



Eq. (12) is completely general, except that the data must be independent. To 
gain some intuition about how the Bayes factor selects models we may now make 
some assumptions about how the global evidence for each model behaves under 
certain kinds of data changes. To begin with, in the case of adding new exclusion 
limits, the best-fit likelihood value of the new data is often very similar in large 
classes of models; specifically, it will be close in value to that for the SM, assuming 
no significant deviations from the SM predictions are observed. An interesting 
situation to consider is thus that in which we set the maximum likelihood value 
for new data to be equal in both model^j Applying this assumption to the Bayes 
factor gives us: 

pi j I j \ ^(W.CMSSM) 

If the CMSSM and SM best fit values for the new data are similar then the Occam 
factors dominate our reasoning process. Models suffering large cuts to the parame- 
ter space become less believable, while those less damaged by the new limits become 
relatively more believable, as one intuitively expects. 

Since this work is devoted to quantifying changes in odds, not odds themselves, 
we evaluate only the Bayes factor B(d2\d{) for various data sets d<i\ the calculation of 
the prior odds in eq. Q is not attempted and is impossible unless one is prepared to 
explore principles for defining measures on the global space of hypotheses -perhaps 
based on algorithmic probability (as advocated by Solomonoff |62| and others)- or 
otherwise justify an 'objective' origin for priors. From a purely subjective Bayesian 
perspective the prior odds can instead be allocated to the reader to estimate from 
their own knowledge base and philosophical preferences, to be modified by the 
Bayes factors we compute. 

To close this section we wish to make an additional observation about our choice 
of the SM as our reference model. It was recently shown in ref. [63] that a model 
in which observable quantities enter directly as input parameters can be considered 
a "puzzle" from the perspective of naturalness considerations. Such a model lies 
at a natural boundary between a fully predictive (or "natural") model (which in 
effect has no free parameters, and for which the evidence collapses to a simple 
likelihood) , and a fine-tuned model (in which small changes to parameters produce 
large changes in predicted observations and for which the evidence due to learning 
the fine-tuning inducing data will be incredibly small, since only a tiny portion of 



4 I.e. in a generic event counting experiment we assume the expected number of signal events at the 
best fit point to be effectively zero. 
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parameter space predicts it correctl}Q. It is argued that such a "puzzle" model 
represents the only sensible reference point against which to measure naturalness. 
The changes in evidence in such a model can be easily computed, if one has enough 
data to define a prior for the observables, and it is argued that these be compared to 
the evidence changes that occur in a model of interest using a Bayes factor exactly 
as we compute; if the Bayes factor favours the "puzzle" model this is an indication 
that the model of interest is not a very natural explanation for the data and drives 
us to believe that a better model should exist. 

There is no reason to restrict this reasoning to only that data usually associated 
with fine-tuning, and as we have defined it our comparison SM is just such a "puz- 
zle" rnode]^} Thus, if the reader prefers, they may interpret our computed Bayes 
factors not as tests of the CMSSM against any specific model, but as measures of 
how much better or worse than the "puzzle" model it predicts the new data (when 
constrained by the baseline data). 

3.1 Interlude: Interpretational issues in Bayesian model 
comparison 

' Unfortunately model order estimation remains a subject of tremendous controversy; 
there is little agreement on what the "best" approach is, and indeed little agreement 
on if there is, in fact, such a thing as a "best" approach. J (A. D. Lanterman) [64] 

Despite its foundational importance to the philosophy of science, there exist many 
open questions in statistical model selection. This is an extensive field of research 
in its own right and we do not wish to venture far into it, however the nature of 
the current work demands that we make some minimal comments. For the sake 
of brevity we restrict the subject of these to interpretational issues associated with 
the Bayesian methods which we employ. 

The expression P(^|data) is immediately of interest. In textbook Bayesian 
scenarios it is usually uncontroversial to interpret this as "the probability that hy- 
pothesis Hi is true, given 'data' is observed" , since Hi is in many cases quite a direct 
proposition about some aspect of reality, such as "Person A's eyes are blue" (given 
they come from population P) or "Production machine X is malfunctioning" (given 
some observed number of defective products). However in the physical sciences 
one generally deals instead with families of statistical models of data, no member 
of which is expected to be a completely "true" descriptions of the real world, es- 
pecially if the models under study are empirical in origin. Models derived from 
physical principles perhaps carry a larger ontological burden but are nevertheless 
still not expected to correspond exactly to reality. To complicate matters, the de- 

5 Note that a very small value for the evidence from learning some data implies a very large amount 
of information was gained about the model. This may sound like a good thing, however it means that 
little was known about the model before this data arrived and so the model was not very useful for 
predicting what that data would be. Bayes factors penalise this failure, however if the information gain 
was sufficiently large then the model may in fact become highly predictive about future data, and may 
thus fare much better in future Bayes factor tests. 

6 The reader may protest that the SM is not just a fine-tuning "puzzle" , it is a very extreme example of 
fine-tuning! However, this is only true if one considers it from a pre-' elect roweak data' perspective. The 
SM presumably suffers a very large Bayes factor penalty for failing to predict the electroweak scale (and 
for this scale being observed very far from, say, the Planck scale, where a priori arguments based on the 
hierarchy problem may place it), however these considerations enter before the 'baseline' data we choose 
for our inference sequence and so do not directly enter our Bayes factors. The complete assessment of 
which model best reflects reality should of course take these matters into account. 
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gree to which a model is expected to reflect reality is context sensitive; for example 
it seems unwise to flatly characterise Newton's laws as "false" given their abundant 
utility, despite the obviously limited domain in which predictions are accurate. 

Despite this difficulty, there is clearly merit in attempting to determine the 
"best'Qof a set of competing models within a given context, although as the quo- 
tation opening this section alludes the definition of "best" presents a problem in of 
itself, with different definitions leading to different metrics for model comparison. 
From the Bayesian perspective of the current work, we expect the quite conservative, 
and pragmatic, interpretation of ref. [65] to sufficiently define our adopted stance on 
these issues; P{Hi |data) is simply the probability that model Hi is the best model 
(of the choices available) to use for data based inference. The context sensitivity is 
thus made obvious (although vague) in that the choice of "best" model depends on 
what future data we would like to predict, and the ontological status of our models 
becomes a side issue. Importantly, the notion of "best" used here is implicitly tied 
to the ability of the model to accurately predict future data; model families with 
many unconstrained free parameters, although they may be able to match future 
data extremely well for some choice of parameters, are also generally uninformative 
as to what that data will actually be in advance, and are thus disfavoured. Indeed 
this characteristic of Bayesian model selection naturally leads to a notion of the 
complexity of a model based on the 'sharpness' of its predictions |66,67], which 
is then satisfyingly penalised in a manner generally consistent with common sense 
notions of the "Occam's razor" principle, with the penalty quantifiable in terms of 
Occam factors. In more general statistical language this aspect of Bayesian model 
selection can be described as a regularisation procedure, i.e. it makes use of extra 
information introduced into an ill-posed problem to prevent over-fitting. 

Other 'common sense' aspects of the complexity of models are unfortunately 
ignored by Occam factors (and by most other regularisation methods), such as the 
number and strength of the assumptions involved in the construction of the models, 
however these considerations are related to foundational questions about the origins 
of prior probabilities of models and so, since we do not consider model priors in our 
current work, are not of immediate interest. We nevertheless note that attempts to 
formalise such notions have been made, with notable efforts including the related 
Minimum Description Length and Minimum Message Length theories of Rissanen 



and Wallace 68,69 , although it appears intractable to apply such theories to the 



automation of model inference in fundamental physics at this stage. 

3.2 State of knowledge transitions 

Here we outline the changes of information that we consider in this paper, and for 
which we compute the corresponding Bayes factors for the CMSSM vs SM hypothe- 
sis test. We take as our initial information a conventional set of experimental data, 
including dark matter relic density constraints, muon anomalous magnetic moment 
measurements, LEP2 direct sparticle mass lower bounds, and various flavour ob- 
servables. The full list and details of the likelihood function can be found in table [2] 
of section [5| Notably, we do not include the LEP2 Higgs mass and cross section 
limits, nor any results from dark matter direct detection experiments or the LHCj^J 
because these are precisely the pieces of data whose impact on the CMSSM we wish 



7 Although in the pure Bayesian paradigm no model is ever 'selected'; the full posterior over the model 
space is always retained. Bayesian decision theory then advocates making decisions based on minimising 
the expected value of a 'loss' function, with the expectation taken over the full space of models. 

8 Except for an early LHCb lower bound on BR(B S — ^ fifi). 
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to assess. To improve the brevity of later references, we name this initial data set 
the "pre-LEP" state of knowledge, to emphasise that the LEP Higgs bounds have 
been removed. 

The shrewd reader will notice that we include many pieces of data in this ini- 
tial set that were not yet measured when the LEP2 Higgs constraints began to 
exclude much of the low-mass CMSSM regions (most notably the WMAP measure- 
ments constraining the dark matter relic density), and that we neglect previous 
Higgs constraints, so our 'initial' knowledge state is not truly representative of the 
experimental situation that existed around say 1998 (when the LEP bound was 
rrih < 77.5 GeV [70] and would not have noticeably constrained our "pre-LEP" 
CMSSM parameter space had we included it). However there is no requirement 
that the analysis be chronologically accurate for meaningful results to be obtained. 
We maintain the rough correspondence simply to ease the interpretation of the re- 
sults. In addition, most extra constraints in the initial set (aside from the WMAP 
data) tend to exclude parts of the CMSSM that the new data would also exclude, 
thus reducing the apparent strength of the latter. 

From this initial data set we add in sequence the LEP2 Higgs constraints and 
XENON100 limits on the neutralino-nucleon elastic scattering cross section to form 
the "LEP+XENON" data set. Next we add the 2011 1 fb" 1 LHC SUSY search 
results to form the "ATLAS-sparticle" data set. Finally, we add the February 2012 
LHC Higgs search results to form the "ATLAS-Higgs" data set. The details of the 
likelihood functions for these new pieces of data are described in section [5j This 
gives us four data sets and three sequential "state-of-knowledge" transitions, each 
of which is described by a Bayes factor. In addition, we compute results using two 
different priors for the CMSSM parameter space (the description of which we leave 
to section 4.1), giving two perspectives on each data change and thus doubling the 



number of data sets and Bayes factors we obtain. 

3.3 Evidences for the Standard Model 

For our purposes, we can consider all the parameters of the SM to be fixed by 
our initial experimental data or otherwise unaffected by the new data, with only 
the Higgs mass undetermined. The new data is also assumed to minimally 
affect any additional dark matter sector. The evidences for the SM for each data 
transition can thus be computed entirely by considering the one-dimensional Higgs 
mass parameter space. This can be shown as follows. 

The above assumptions allow us to separate the parameters and available data 
into three groups: (1) initial data do which highly constrains the Standard Model 
parameters (j) but less strongly constrains m^; (2) data cIq which constrains only 
the dark sector parameters u, and whose effect on (j) is negligible relative to do] and 
(3) 'new' data d new constraining only the Higgs mass, with negligible effect on (j) 
relative to do, and no impact on the dark sector parameters uj. If we assume that the 
'fundamental' prior for uj is independent of that for and 0, i.e. P(mh, 0, cj|SM) = 
P(m/ l , 0|SM)P(cj|SM), and that the three sets of data are statistically independent, 
then the evidence associated with the new data d new can be written as 

73/7 17 7 qi\ /r\ P(d new ,do,dn\SM) 

P(d new \d ,d n ,SM) = p{doMSM) (15) 

= J dm /t d^<L;P(d Ttew |SM, m h , <f>)P(d \SM, m h , <f>)P(dsi\SM, <f>, co)P(m h , <f>, lv\SM) 
J dm h d<f>dujP(do\SM, m h , <f>)P(d n \SM, <j>, co)P(m h , <j>, u\SM) 

If do sufficiently strongly constrains the SM parameters (except m/j) to <fr' then to a 
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good approximation P(do|SM, m^, ft oc 5 (ft — ft)f(mh), where f(rrih) describes the 
variation of the do likelihood in the direction, and the proportionality constant 
divides out in the ratio. The <j) integral is thus removed and the remaining integrals 
are separable. The integral over the dark sector parameters uo is identical in the 
numerator and denominator and thus vanishes, as does the prior density P(ft\SM) 
(resulting from expanding P(m/ l , 0'|SM) as P(m/ l |SM, ft)P(ft\SM), evaluated at 
ft due to the delta function), leaving us with 

PM u A Qi |\ fdm h P(d new \SM, m h , ft)f(m h )P(m h \SM, </>') 

P{anew\ao, an, blVLJ = ~- — — — — (lb J 

J dm h f(m h )P(m h \SM, ft) 

We are free to choose the normalisation of /(ra^), and it is convenient to choose it 
such that fdrrihf(rrih)P(rnh\SM,(f) f ) = 1, so that /(m/ l )P(m/ l |SM, (j)') corresponds 
to the posterior probability density for once do is considered, i.e. P(rrih\do, SM, r 
This density becomes the prior for the consideration of d new . The evidence associ- 
ated with learning d newi starting from do and ofo, is thus shown to be the relatively 
straightforward integral 

P(d new \d ,d n ,SM) = Jdm h P(d new \SM,m h ,ft)P(m h \d ,SM,ft). (17) 

as we intuitively expect. Importantly, this evidence is independent of the details 
of both the dark sector theory and the constraints cfo, so long as the theory meets 
our criteria of not significantly affecting the predictions for d newi nor is affected by 
the value of m^^j Any sufficiently decoupled dark sector satisfies this requirement. 



We now evaluate eq.(17). The do relevant for constraining are electroweak 
precision measurements, so we may build our prior P(m/ l |do 5 SM, ft) — f(rrih) 
x P(rrih |SM, ft) based on these. Taking the most conservative A\ 2 curves from 
figure 5 of ref. [7l] as our electroweak constraints we reconstruct the corresponding 
likelihood function /(m^), and multiply this by a 'fundamental' prior P(m/ l |SM, ft) 
flat in log m/P^j Although this is done numerically it yields a prior clos^Jto a broad 
Gaussian (in log space) centred on — 90 GeV with a log 10 width of about 
0.15, i.e. m h = 90^ GeV. 

If the new data d new is the LEP2 likelihood function described in table [2] 



(let us call this (Ilep), then eq.(17) is now straightforward to evaluate numerically. 
Its value alone is not meaningful because the likelihood function is only defined up 
to a constant (which divides out in the Bayes factor), however if we divide out the 
maximum likelihood value we recover the corresponding Occam factor, which we 
find to be 0.284, or about 1/3.5. We have checked that choosing a flat fundamental 
prior for makes little difference to this result. We consider the corresponding 
effects on the CMSSM in section [6j however it is useful to mention here that the 
maximum likelihood values for both the SM and CMSSM for this data are equal 
(since our simple model of the limit assumes the likelihood to be maximised for 
the background-only hypothesis), so the Occam factors themselves contain all the 
information about which model the Bayes factor prefers. A more careful analysis of 
the LEP data would allow the CMSSM to receive a slight likelihood preference since 
it is has more parameters than the SM and can in principle achieve a better fit to 



9 Within the range of values compatible with d new , i.e the dark sector theory is permitted to 
exclude values of which are also well excluded by d new . 
10 For a scale parameter this is the Jeffreys prior. 

11 These A% 2 curves are almost quadratic in logm^, implying a close to Gaussian likelihood function, 
however we have digitised the most loose boundaries of the displayed curves to be conservative. As a 
result the likelihood function we reconstruct has a flat maximum from ~ 80 GeV to ~ 100 GeV. 
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any observed deviation from the expected background, however since no significant 
excess was seen at LEP this effect will be small. 



In addition to the evidence P(dLEp\do, SM), the computation of eq.(17) also 



produces for us (via Bayes' theorem) a new posterior distribution over rn^. which 
incorporates both do and d^EP (with cIq having had no impact): 

p , , qM x P(d LEP \SM,m h )P(m h \d ,SM) 
P(m h \d LEP ,d ,SM) = P{dLEpldo ^M) (18) 

(for brevity we drop the conditionals on as it is fixed from here on, and on ofo, 
because our results were shown to be independent of it). This is the prior for the 
second iteration of our learning sequence, in which we consider the addition of the 
ATLAS sparticle search results. These of course do not affect the Standard Model 
parameters, and our assumptions about the nature of the dark sector demand that 
it be similarly unaffected. So the evidence in the SM for this data change can be 
safely set to 1. 

Finally, we consider the addition of the recent LHC Higgs search results. Since 
the sparticle searches had no impact the prior for this step is unchanged in form from 



eq.([18j). As we shall discuss further in section \5.3\ we constrain the CMSSM using 
only the results from the ATLAS h -> 77, h -> ZZ 4/ and h -> WW 2l2u 
search channels [72||74| , as these channels both dominate the constraints on the 
lightest CMSSM Higgs and are the only ones for which ATLAS provide signal best 
fit plots, which we require to perform our likelihood extraction. CMS do not provide 
such plots for all channels so we are unable to incorporate the CMS results at this 
stage. We constrain the cross sections for each of these channels separately in the 
CMSSM likelihood function since the factor by which they differ from the Standard 
Model prediction is not uniform across all channels, as is assumed in the ATLAS 
and CMS combinations. For the SM evidence computation it would be optimal 
to include extra channels which can more powerfully exclude higher Higgs masses, 
however the strength of the 125 GeV excess in our chosen three channels is already 
sufficient to very strongly disfavour such Higgs masses, such that including these 
extra channels would negligibly improve our analysis. 

In figure [T] we show the prior over the SM Higgs parameter derived from elec- 
troweak precision measurements, with LEP and ATLAS Higgs search likelihood 
functions overlaid. The LEP likelihood function is simply taken as a hard lower 
limit at 114.4 GeV, convolved with a 1 GeV Gaussian experimental uncertainty (as 
described in table [2]). The ATLAS Higgs search likelihood function is reconstructed 
from the February 2012 combined Higgs search results [75] using the method de- 
scribed in section |5.3| Applying each of these likelihood functions in sequence we 
compute Occam factors of 0.284 and 0.02 respectively. 

It is worth a final comment that we have not folded in earlier LEP Higgs limits 
into the prior for the SM Higgs mass; for example the upper limits of around 80 GeV 
that existed in 1998. We have done this to avoid an arbitrary decision about exactly 
which limits to include, and because neglecting them only weakens the apparent 
damage that LEP did to the CMSSM. This occurs because CMSSM model points 
predicting Higgs masses of below 80 GeV are not common nor have particularly 
high likelihood in our "pre-LEP" CMSSM data set and so do not occur in most 
of the effective prior which arises from that data set, while a very sizable portion 
of the SM Higgs mass prior we have just constructed is below 80 GeV. Therefore, 
were we to include such a limit, it would increase the apparent damage done by 
the 114.4 GeV LEP limit to the CMSSM (relative to the SM), so leaving it out is 
a conservative choice. The impact on the corresponding Bayes factor can be fairly 
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Figure 1: The prior over the SM Higgs mass parameter derived from electroweak preci- 
sion measurements (green), with LEP (blue) and ATLAS (red) Higgs search likelihood 
functions overlaid. The prior and likelihood functions are scaled against their maximum 
values. 



easily estimated anyway by considering eq. (13), as follows. The amount of "pre 



LEP" CMSSM posterior that would be affected is fairly negligible so we can ignore 
it in a rough estimate, while the amount of SM Higgs prior that would be cut off 
can be seen from figure [l] to be about 1/3. The 114.4 GeV limit would thus have its 
SM Occam factor increased from about 0.3 to about 0.5 (weakening it), and since 
the other components of the Bayes factor remain unchanged the effect would be 
about a 5/3 boost in odds towards the SM, which, as we shall see in section [6| is 
of negligible importance. 



4 Evidences for the CMSSM 

To determine the CMSSM part of our Bayes factors we compute the CMSSM global 
evidence under each of the data sets described in section [372], using two contrasting 



prior distributions for the parameters (see section 4.1 for details). This requires the 
numerical mapping of the CMSSM global likelihood function for each data set (the 
details of which we discuss in section [5]). To perform this mapping we use the pub- 
lic code MultiNest v2.12 [76j[77], which implements Skilling's nested sampling 



algorithm |78 
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To compute the CMSSM predictions at each parameter space 

We 



point we first generate the particle mass spectrum using ISAJET v7.81 81 



then pass the spectrum to mi cr Omegas v2.4.Q |82j|84j to compute the neutralino 
relic abundance, muon anomalous magnetic moment, spin-independent proton- 
neutralino elastic scattering cross section, and precision electroweak variable Ap. 
We use SuperlSO v3.1 |85, 86 to compute a number of flavour observables (the 
full set of likelihood constraints imposed is listed in table [2]). We also estimate 
on a yes/no basis whether a given point can be considered excluded by ATLAS 
direct sparticle searches, using a machine learning technique which we describe in 

which we use to 



section 



5.2 



Finally, the spectrum is passed to HDECAY v4 . 43 87 



12 Aside from nested sampling and several variants of Markov Chain Monte Carlo methods, the list of 
techniques used to scan the CMSSM has expanded in recent years to also include genetic algorithms and 



neural networks 79 80 
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compute the cross section ratio ctcmssm/^sm for the following processes: 

gg 77, 

gg —th—t ZZ — » 4/, and 

gg^rh^r WW -> 21 + 2v. (19) 

These cross sections we constrain separately using the December 2011 - February 
2012 ATLAS Higgs search results [72}|74] . MultiNest then guides the scan through 
a large number of sample points, returning a chain of posterior samples and the 
evidence. 

To ensure that the likelihood function is sampled densely enough to guarantee 
highly accurate evidence values, we run MultiNest with parameters guided by 
the recommendations of ref. [88], in which MultiNest is configured to sample the 
likelihood function to the accuracy required for frequentist analyses. Since our 
analysis is Bayesian we do not require as detailed information as frequentist scans 
in the vicinity of the maximum likelihood points, so we drop the recommended 
number of live points from 20/c to 15/c and relax the convergence criterion from 
tol = 10 -4 to tol = 0.01 to reduce the computational demand. Additionally, we 
cluster in three dimensions (Mo, M]y 2 and tan/3) and set the efficiency parameter 
efr to 0.3. Finally, we treat the top quark mass m t as a nuisance parameter 
(with the rest of the Standard Model parameters set to their central experimental 
values), so the dimensionality of the scanned parameter space is five. The above 
MultiNest settings result in about 10 7 evaluations of our likelihood function per 
run and posterior chains of about 2.5 x 10 5 good model points. The total number 
of likelihood evaluations over the whole project exceeded 10 8 . 



4.1 Priors and ranges 

The shape of the prior P{9\ CMSSM) reflects our relative belief in different parts 
of the parameter space before learning the relevant experimental information. By 
selecting multiple priors we can analyse how a representative set of subjective beliefs 
about the CMSSM should be modified by new data. We now describe the priors 
we use and explain our choices. 

We allow the top quark mass to vary since, of the Standard Model parameters, 
its experimental uncertainty allows the largest variation in the CMSSM predictions. 
Since its value is to a large degree fixed by experiment we are able to set its prior to 
be a Gaussian with the experimental central value and width of 172.9=hl.l GeV [89] . 

To reduce the computational complexity of the problem we have only scanned 
the fi > branch of the CMSSM. This is not optimal, however it almost certainly 
does not greatly affect our inferences, because the n < branch is already strongly 
disfavoured by the data in our "LEP+Xenon" set by a Bayes factor of around 20-60 
1 56^} The /i < branch of the "pre-LEP" dataset is therefore less disfavoured than 
this, and so the fraction of parameter space disfavoured by this first transition must 
be larger than we compute, making our estimated Bayes factor for it conservative. 
In subsequent transitions the volume of parameter space left viable in the fi < 
branch would be this factor of 20-60 smaller, and so changes to it would contribute 
by the same factor less to our total Bayes factors, rendering it quite unimportant 
for those transitions. 

13 The authors estimate these Bayes factors using both flat and log priors; here we refer to the log prior 
results only since we do not use flat priors. In addition, the da^ constraint is shown to strongly drive the 
preference of /i > so if the validity of this constraint is questioned (we consider the effects of removing 
it in section [6|) then the impact of ignoring the \i < branch may also merit revisitation. 
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4.1.1 Logarithmic prior 



Based on naturalness arguments there is a strong belief that all CMSSM parameters 
with mass dimensions, {Mo, M1/2? should be low. A flat prior distribution for 
these parameters would strongly conflict with this belief; with a flat prior a mass 
parameter would be considered 10 times more likely to be between 1 TeV and 10 TeV 
than between 100 GeV and 1 TeV, increasing 10 fold again each order of magnitude, 
which we consider undesirable. In contrast logarithmic priors favour neither low nor 
high scales, and so can be considered to represent a 'neutral' position on the issue 
of naturalness. Such a prior is flat in log(0), resulting in P{9 \ CMSSM) oc 1/9. The 
log prior has the additional mathematical attraction of being the Jeffreys prior |90 
for a scale parameter, i.e. it is minimally informative in the sense that it maximises 
the difference between the prior and the posterior for such parameters. In our case 
Mo and Mi/ 2 are assigned independent log priors, while A$, and tan/3 are left with 
flat priors. We make the latter choices because Ao ranges over positive and negative 
values and so is more appropriately considered to be a location parameter (for which 
the Jeffreys prior is the flat distribution) and because tan/3 varies over only one 
order of magnitude. Each of these beliefs about the parameters are considered to be 
statistically independent, so the full prior is obtained simply by multiplying them 
all together: 

1 

MoM 1/2 ' 

Numerous studies of the CMSSM have already been performed using this prior [28[ 



P(M , Mi/2,4), tan /3| CMSSM) oc — — . (20) 



30,57,88], making it a good standard prior to consider. Such studies often also 
employ a flat prior in the mass parameters however we do not, for the reasons 
explained above, preferring to save our CPU time for the natural priors discussed 
below. 



4.1.2 Natural prior 

Naturalness is a theoretical consideration which can be used to set the shape of 
the prior distributions within the CMSSM, and which can be quantified in terms 
of fine-tuning. Several measures of the degree of fine-tuning in a model exist [41] , 
but probably the most well known is the Barbieri-Guidice measure [91] 



A 



d In m 2 z 



d\n9 



(21) 



which quantifies the sensitivity of the Z boson mass to the variation of the param- 
eter 9. In ref. [53], and later [24][44j[54], it was shown that a prior incorporating 
this measure to penalise high fine-tuning can be constructed from purely Bayesian 
arguments. 

The key idea is relaxing the usual requirement of the CMSSM that the /x param- 
eter is fixed by the experimental value of mz through the Higgs potential minimi- 
sation conditions and instead incorporating mz into the likelihood function. One 
then starts from flat priors over the natural parameter set Mo, M1/2, Ao, B and /i. 
Using the observed mz to marginalise over ji and transforming to the usual CMSSM 
parameter set introduces a Jacobian term penalising high tan /3 and a fine-tuning 
coefficient penalising high fi values, giving us the natural (or "CCR") prior [92] 

P„ e (M ,M 1/2 ,^, tm/S | CMSSM) oc t J%^^ . <*) 
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Here B\ ow is the low energy value of the B parameter and fiz is the fi value re- 
quired to produce the correct Z mass. Operationally, we implement this prior by 
scanning the conventional parameter set with a flat prior and multiplying the above 
expression into the likelihood function. 14 

The above prior does not fully implement the Barbieri-Guidice measure because 
it only considers the fine-tuning of the \i parameter. In ref. |54 an extended version 
of this prior is constructed which also considers the tuning of the Yukawa couplings, 
and in refs. (93j|94j a generalisation to the full parameter set is considered, but 
we choose to focus only on the simpler version in this work, since it captures a 
large amount of the fine-tuning effect and can be computed analytically once the 
spectrum generator (ISA JET) has run. 

4.2 Effect of the parameter ranges on Bayes factors 

In many recent studies of the CMSSM, only relatively low mass regions of the 
parameter space have been considered, generally regions not much larger than < 
Mo, Mi/2 < 1 TeV. This is in part motivated by naturalness arguments, in part 
by the generally lower likelihood outside this region (largely driven by a poorer fit 
to 5a M ), and perhaps largely because the LHC SUSY search limits will not reach 
deeper into the parameter space than this for several years yet. Ideally, since we 
would like to consider changes in the total evidence for the CMSSM, it is desirable 
to consider the entire viable parameter space, since the more viable space that 
exists outside the LHC reach, the less the CMSSM will be appear to be harmed by 
it. However, it is extremely difficult to thoroughly scan the CMSSM out to very 
large values of Mo and Mj/2 due to the computational expense of obtaining reliable 
sampling statistics. In addition, our study is primarily concerned with obtaining 
Bayesian evidence values, which involve integrals over the parameter space and so 
are more vulnerable to poor sampling than the posterior. 

Due to this, we focus only on the low mass region of the CMSSM. The apparent 
impact of the LHC on the CMSSM will thus be increased, though it will be a faithful 
estimate of the damage done to the low mass region. Importantly, the change this 
restriction makes to the final Bayes factors will be determined by the volume of 
the initial posteriors (those resulting from the "pre-LEP" data) that we neglect, 
not the full change of volume of the scan priors, as occurs for the global evidence. 
This is because our incremental evidences are ratios of global evidences, so factors 
due to the scan prior volume divide out. Indeed, were our scans to contain 100% 
of this initial posterior, then further increases in the scan prior volume would have 
no effect at alf^l 

Finally, we should consider the bias that exists in our assessment of the damage 
done to the CMSSM due to our choice of the SM as the alternate model. There of 
course exist numerous models which may be of more direct interest as alternatives 
to the CMSSM, which suffer more damage than our SM-like alternate does due 
to their larger parameter spaces, and comparing the CMSSM to these we would 
conclude that the posterior odds for it were better than when compared to our SM- 
like model. This consideration forms part of our motivation to present our results 
in terms of both Bayes factors and the constituent likelihood ratios and Occam 
factors, as we hope this allows the reader to more easily understand how changes 

14 We do this because P e ff requires renormalisation group running to be evaluated, i.e. our spectrum 
generator needs to be run before we can evaluate P e ^. 

15 In practice a larger scan volume will decrease the scan resolution and reduce the accuracy of results, 
so prior volume dependence would still exist in this indirect form. 
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Priors 


Name 


PDF 


Log 

Natural 


l/(M M 1/2 ) 
tan 2 /3 - 1 .Blow 


tan 2 /3(l + tan 2 /3) /i Z 


Ranges 


Parameter 


Range 


M 


10 GeV - 2 TeV 


Mi/a 


10GeV-2TeV 


A 


-3TeV-4TeV 


tan j3 


0-62 


sign(/i) 


+1 


ra t 


172.9 ± 1.1 GeV 



Table 1: Summary of the priors and ranges used in this study. The displayed PDFs for 
both priors are multiplied by a Gaussian for m t with mean and width specified by the 
values adjacent to m t in the table. 



in alternate model will affect our inferences. We will return to this discussion when 
we present our results in section |6j 

A summary of the priors and ranges used for this study are presented in table [T] 



5 Likelihood function 

We now detail the experimental data which goes into our likelihood function. Pri- 
marily this is summarised in table [2| Each of these components is considered to 
be statistically independent, so that the global likelihood function is simply the 
product of them: 

^Global = II ^ = II P ^\ ' CMSSM). (23) 

i i 

The 2011 constraints from the XENON100 experiment and the LHC Higgs and 
sparticle searches cannot be implemented via likelihood functions simple enough to 



list in a table, so we explain our treatment of them in sections |5. 1| through 5.3 



5.1 XENON100 limits 

The likelihood contribution from XENON does not yet have a significant impact on 
the CMSSM evidence so we have opted to simply model the likelihood as an error 
function of the WIMP-nucleon spin-independent elastic scattering cross section, 
which varies with the WIMP mass. Our likelihood function for the cross section 
(°"fo-p) is derived from the 90% confidence limits published by the XENON100 
experiment in figure 5 of ref. |103| . This limit is presented as a function of WIMP 
mass. We fit the likelihood function with an error function such that it reproduces 
the correct 90% C.L. and the correct apparent significance of the upper edge of 
the la sensitivity band, based on the maximum likelihood ratio method, using a 
similar procedure to that used in ref. |106| to estimate their likelihood function for 
a CMS multi-jet+^r search, which we summarise below. 
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Observable 


Measured value 


Computed by 


Sources 



Gaussian likelihoods 



Ap 

5a^ 

BR(b — > 
BR(B - 
A _(B- 

Rl23 

BR(D S tv) 
BR(D S /iz/) 



57) 

• rz/) 

* iP 7 ) 

^D Q ev J 



0.0008 ±0.0017 
0.1123 ±0.0035 ±10% 
3.353 ± 8.24 [xlO- 9 ] 
3.55±0.26±5%[xl0- 4 ] 
1.67 ± 0.39 [xlO" 4 ] 
0.416 ±0.128 

0.029 ± 0.039 

1.004 ±0.007 
0.0538 ± 0.0038 
5.81 ±0.47 [xl0~ 3 ] 



micrOmegas 2.4.Q 
micrOmegas 2.4.Q 
micrOmegas 2.4.Q 
SuperlSO 3.1 
SuperlSO 3.1 
SuperlSO 3.1 

SuperlSO 3.1 

SuperlSO 3.1 
SuperlSO 
SuperlSO 



3.1 
3.1 



95 
96 
97 
97 
98 



99 



100 
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Limits (erf) 



m~ g 


> 289 ± 15 GeV (LEP2) 


ISAJET 7.81 




101 




m h 


> x ± 3 GeV a (LEP2) 


ISAJET 7.81 




102 
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Limits (hard cut) 



Other LEP2 direct sparticle mass 95% C.L.'s 


ISAJET 7.81 




101 




Special cases 


Xo-P 

BR(B S 

SUSY searches 


See text (XENON100) 
< 1.5 x 10- 8 (LHCb) 
See text (LHC) 
See text (LHC) 


micrOmegas 2.4.Q 
SuperlSO 3.1 
ISAJET 7.81, HDECAY 4.43 
ISAJET 7.81, Herwig++ 2.5.2, 
Delphes 1.9, PROSPINO 2.1 




103 
104 
72f 
105 


12 
75 



' x determined from figure 3a of ref. |102| for 
x = 114.4 GeV. 



each point. For nearly all CMSSM points 



Table 2: Summary of the likelihood functions and experimental data used in this analysis. 
Gaussian likelihoods: Likelihoods are modelled as Gaussians; where two uncertainties are 
stated the first arises from experimental/ Standard Model sources, while the second is an 
estimate of the theoretical/computational uncertainty in the new physics contributions 
(and these are added in quadrature), otherwise the latter uncertainty is assumed to be 
small and treated as zero. Limits (erf): The listed central values are estimated 95% C.L.'s, 
and are used to define a step function cut, which is convolved with the stated Gaussian 
estimate of the total (experimental and computation-based) uncertainty. Limits (hard 
cut): Step function likelihoods centred on the cited 95% C.L.'s are used. Special cases: 
For details see sections 5.1 though 5.3 (and footnote 12 for B s —> /x H 



> )• 



1 Section 'Electroweak model and constraints on new physics', p. 33 eq. (10.47). We take the larger 
of the 1 sigma confidence interval values. The full likelihood function is actually highly asymmetric and 
slightly disfavours values close to the Standard Model prediction, which we are effectively ignoring. 

2 Table 1 (WMAP + BAO + HO mean). Theoretical uncertainties are not well know so we follow the 
estimates of ref. 30 . 

3 Table 10 (Solution B). 

4 Table 129 (Average). 

5 Table 127. 

6 Page 17, uncertainties combined in quadrature. 
7 Table 1 (R value) 
8 Eq. (4.19). 

9 Figure 68, p. 225 (World average). 
10 Figure 67, p. 224 (World average). 
11 Figure 3a, p. 24. 

12 Figure 8. We use the full CL S curve rather than simply the 95% confidence limit. Working backward 
from the CL S values given by the curve, assuming them to be instead CL s +b values, we determine the 
corresponding likelihood function which would generate these values (assuming a chi-square distributed 
test statistic). CL S intervals over-cover so this procedure is conservative. 
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XENON use the profile likelihood ratio test statistic 



Q = -2 log A = -2 log -fj = -2 log iff- 24 



to derive their exclusion limits, where mj. cr^ and <t^ are the hypothesised 
WIMP mass, spin-independent WIMP-proton scattering cross section, and best 
fit value of the latter for each mass slice, respectively. All nuisance variables are 
profiled over and limits are derived on the cross section for each fixed mj, so the 
resulting profile likelihood ratio has one degree of freedom and Q is asymptotically 
distributed as f(Q\cr;mx) = Xk=i(Q) (which XENON100 have confirmed is true 
to a good approximation via Monte Carlo |107| ). The cross section is proportional 
to the mean signal event rate \i for each mx slice, so we may use the asymptotic 
expressions of |108| to express Q in terms of \i as 

where jl is the best fit signal event rate for some observed data, which is normally 
distributed with standard deviation 

XENON report the observation of 3 events in their signal region, with an 
expected background of 1.8 ± 0.6 events, so a = 0.6 and jl = 1.2. The up- 
per 90% confidence limit is drawn on the contour on the (mx^cr^) plane on 
which the predicted mean event rate drops to the level producing Q such that 
p s = JdQf(Q\a;m x ) = 0.10 or Q = 2.71. To fit our erf model likelihood we 
require a second contour of Q, and the expected+la limit is a convenient choice. 
On this contour, a hypothetical observation of 1.8 + 0.6 = 2.4 events is assumed, 
which produces a best fit signal mean of jl — 0.6. Again the 90% confidence 
limit is drawn where Q drops to 2.71, which occurs at = jl + \j2.7\a « 1.59. 
Knowing the predicted signal rate on this contour now allows us to infer the 



value of Q on this contour given the actual observed data, i.e. from eq. (5.1) 
Q « (1.59 - 1.2) 2 /0.6 2 « 0.417. Our erf likelihood is fitted to reproduce these 
contours for each mx slice, thus producing an approximation of the full likelihood 
function 
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We ignore the variation of a with the predicted signal rate as it is small for small signal. 



17 Actually the CL S method is used so the limit is drawn where p = — Pb) = 0.1 109 , but this 

correction weakens the limit so it is conservative to ignore it and in this case makes little difference 
anyway, given our other approximations. 

18 In section 5.3 we construct the ATLAS Higgs search likelihood function using almost identical tech- 
niques, but argue that each fitted slice needs to be normalised relative to the others using the likelihood 
of the best fit point on each slice. This occurs because the best fit point of each slice lies a varying 
number of standard deviations from the zero signal point (zero cross section), which we know to have 
the same likelihood for every slice. A similar normalisation is in principle required to recover the true 
likelihood function computed by Xenon, however the variance of the limit appears to be approximately 
Gaussian in the logarithm of the cross section, making extrapolation of the likelihood to the zero cross 
section point extremely unreliable. In addition, the reconstruction method we use for the ATLAS Higgs 
search likelihood relies on plots of the signal best fit against mx, whereas here we use a plot of the 90% 
confidence limit. Performing the extraction using the limit curve requires more assumptions than a best 
fit curve, so combined with the logarithmic difficulty we judge that this technique would produce poor 
results, and so we prefer to stick with the simpler technique described. The Xenon limit turns out to be 
of very minor importance to our final inferences anyway so we are not concerned with small errors in our 
reconstructed likelihood. In hindsight we expect that even simply applying a hard cut at the observed 
Xenon limit would negligibly affect our inferences. 
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5.1.1 Hadronic uncertainties 



The above procedure is simplistic, but it gives us a good enough estimate of the 
experimental uncertainty associated with cr^ 1 for our purposes. In addition to 

X.0 y 

this, we fold in an estimate of the associated theoretical uncertainties, assumed 
to be dominated by the uncertainties in the strange quark scalar density in the 
nucleon, in turn due mainly to the experimental uncertainty in the 7r-nucleon a 
term, = 1/2 (m u + md)(N\uu + dd\N). Numerous estimates of this quantity 
exist (59 ± 7 [IlO], 79 ± 7 [111 , -45 [Tl2] , 64 ± 8 [Tl3) [MeV]) and it is not clear 
which are the most reliable so we opt to use a recent value on the low end of the 
spectrum based on lattice calculations, with a wide u ncertainty (39 ± 14 JTl4p^| 



with do = 1/2 (m^ + rrid){N\uu + dd — 2ss\N) = 36 ± 7 116-119 ) as this produces 
low <rf~l_ n predictions and so a conservatively weak XENON100 constraint. 

X0 y 

The computation of &^_ p is performed by micrOmegas v2.4.Q, and it accepts 
E^ as an input parameter, along with ctq- To estimate the uncertainty in the 
computed cross section due to these quantities, they were first used to estimate the 
corresponding parameters = (N\m q qq\N) /tun and their uncertainties (fol- 

lowing |120| ), which micrOmegas computes internally and uses in its computation 
of erf 7 n . We find these to be = 0.016 ± 0.007, & ] = 0.023 ± 0.010 and 

X0~P J J-u ' J J-d 

(v) 

frp = 0.039 ± 0.026, in close agreement with the values micrOmegas computes 
internally from our chosen E^ and a$. We have modified micrOmegas so that 
our computed uncertainties on the are then propagated alongside the 

themselves in the computation of erf 7 and used to estimate the uncertainty on 

xo y 

a ^Q- p f° r each model point. This uncertainty is then added in quadrature to the 
width of the o~^_ p erf likelihood function (i.e. convoluted into it). 



5.1.2 Astrophysical uncertainties 

In our model of the o-~,^_ p likelihood function, we do not rigorously consider the 
effects of varying the astrophysical assumptions that XENON have made in their 
construction of their confidence limits. In their analysis XENON assume WIMPs 
to be distributed in an isothermal halo with vq = 220 km/s, galactic escape velocity 
v esc = 544+46 km/s, and a density of p x = 0.3GeV/cm 3 , and we cannot change 
these without developing a model of the likelihood function based directly on the 
event rate observed by XENON100, as is done in ref. [25] and [30], for example. 
We have opted not to do this as (30l shows that marginalising over a range of plau- 
sible values near the nominal choice makes negligible difference to the impact the 
XENON100 experiment has on the CMSSM, and we prefer to avoid the additional 
increase in the dimensionality of the problem. 



5.2 1 fb 1 LHC sparticle searches 



In late 2011 the ATLAS and CMS experiments [121[|122 up dated their s earches 
for supersymmetric particles using the 2011 lfb -1 



105 



123 



129 



Data 



data set 

collected from proton collisions at the Large Hadron Collider at = 7TeV are 
analysed in a variety of final states, none of which show a significant excess over the 
expected Standard Model background. As the LHC is a proton-proton collider, one 
expects to dominantly produce coloured objects such as squarks and gluinos, whose 



19 From eq. (5), using the suggested <j s 
m s /(2(m u + m d )) = 26 ± 4 [89]. 



50 ± 8 MeV and o x = 47 ± 9 MeV 115 , with m s /mi 
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Region 


Rl 


R2 


R3 


R4 


RHM 


Number of jets 


> 2 


> 3 


> 4 


> 4 


> 4 


£™ iss (GeV) 


> 130 


> 130 


> 130 


> 130 


> 130 


Leading jet pr (GeV) 


> 130 


> 130 


> 130 


> 130 


> 130 


Second jet pt (GeV) 


> 40 


> 40 


> 40 


> 40 


> 80 


Third jet p T (GeV) 




> 40 


> 40 


> 40 


> 80 


Fourth jet p T (GeV) 






> 40 


> 40 


> 80 


A0Get, ^ iss ) min 


> 0.3 


> 0.25 


> 0.25 


0.25 


> 0.2 


m e ff(GeV) 


> 1000 


> 1000 


> 500 


> 1000 


> 1100 



Table 3: Selection cuts for the five ATLAS zero lepton signal regions. A0(jet, p^ lss ) min is 
the smallest of the azimuthal separations between the missing momentum p™ lss and the 
momenta of jets with pt > 40 GeV (up to a maximum of three in descending pt order). 
The effective mass m e ff is the scalar sum of E^ 1SS and the magnitudes of the transverse 
momenta of the two, three and four highest Pt jets depending on the signal region. In 
the region RHM, all jets with p T > 40 GeV are used to define m e ff. 



Region 


Rl 


R2 


R3 


R4 


RHM 


Observed 
Background 
a x A x e (fb) 


58 

62.4 ±4.4 ±9.3 
22 


59 

54.9 ±3.9 ±7.1 
25 


1118 
1015 ±41 ±144 
429 


40 

33.9 ±2.9 ±6.2 
27 


18 

13.1 ± 1.9 ±2.5 
17 



Table 4: Expected background yields and observed signal yields from the ATLAS zero 
lepton search using 1 fb -1 of data |105| . The final row shows the ATLAS limits on the 
product of the cross section, acceptance and efficiency for new physics processes. 



inclusive leptonic branching ratios are relatively small, and hence the strongest 
CMSSM exclusions result from the ATLAS searches for events with no leptons and 
the CMS searches for sparticle production in hadronic final states. The ATLAS 
and CMS limits have a similar reach in the squark and gluino masses, and here we 
consider only the ATLAS zero lepton limits for simplicity. Recent interpretations 
of LHC limit results can be found in [^[3^ [l0^pOHT32] . 

The ATLAS signal regions were each tuned to enhance sensitivity in a particular 
region of the Mq-M 1 / 2 plane. Events with an electron or muon with pt > 20 GeV 
were rejected. Table [3] summarises the remaining selection cuts for each region, 
whilst table [4] gives the observed and expected numbers of events. These numbers 
were used by the ATLAS collaboration to derive limits on a x A x e, where a is 
the cross section for new physics processes for which the ATLAS detector has an 
acceptance A and a detector efficiency of e. These results are also quoted in table [ij 

The ATLAS collaboration have used the absence of evidence of sparticle pro- 
duction in 1 fb -1 of data to place an exclusion limit at the 95% confidence level 
in the Mq-Mi/ 2 plane of the CMSSM for fixed Ao and tan/3, and for fi > 0, and 
all previous phenomenological interpretations of this limit in the literature have 
also ignored the Aq and tan/3 dependence. Ref. [28], for example, finds a negligi- 
ble dependence of the limits on Aq and tan/3. However, as we shall see later this 
conclusion no longer holds for the present limits which are considerably stronger. 
Given that we wish to work in the full space of the CMSSM, we do not use the 
ATLAS exclusion limit directly, but instead simulate our own signal events for a 
points in the full CMSSM using standard Monte Carlo tools coupled with machine 
learning techniques to reduce the total simulation time. 
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This section is structured as follows. Firstly, we explain and validate the tools 
we use to go from a set of CMSSM parameters to a signal expectation. We then 
examine why it is important not to neglect Aq and tan (3 in LHC limits, by showing 
a class of model that fits the ATLAS data extremely well but would be missed if 
one were to assert Ao = 0. Finally we address the fact that, when updating the 
posterior distributions obtained pre-LHC with the ATLAS results, it is not feasible 
to simulate every point in the posterior. We therefore spend the remainder of this 
section developing a fast simulation technique derived by interpolating the output 
of a much smaller number of simulated points using a Bayesian Neural Network. 



5.2.1 Simulating the ATLAS results 

Given a signal expectation for a particular model, one can easily evaluate the like- 
lihood of that model using the published ATLAS background expectation and ob- 
served event yield in each search channel. By simulating points in the full CMSSM 
parameter space, we can therefore investigate the LHC exclusion reach, provided 
that we can demonstrate that our simulation provides an adequate description of 
the ATLAS detector. 

In this paper, we use ISAJET 7.81 [8l] to produce SUSY mass and decay 
spectra then use Herwig++ 2.5.2 [133] to generate 15,000 Monte Carlo events. 
Delphes 1 . 9 |134| is subsequently used to provide a fast simulation of the ATLAS 
detector. The total SUSY production cross section is calculated at next-to-leading 
order using PR0SPIN0 2.1 1 135 1, where we include all processes except direct pro- 
duction of neutralinos, charginos and sleptons since the latter are sub-dominant. 
The ATLAS set-up differs from this only in the final step of detector simulation, 
where a full, Geant 4-based simulation |136| is used to provide a very detailed 
description of particle interactions in the ATLAS detector at vast computational 
expense. 

It is clear that the Delphes simulation will not reproduce every result of the 
advanced simulation. Nevertheless, one can assess the adequacy of our approxi- 
mate results by trying to reproduce the ATLAS CMSSM exclusion limits. We have 
generated a grid of points in the Mq-Mi/ 2 plane using the same fixed values of 
tan/3 = 10 and Ao = as the published ATLAS result. We must now choose a 
procedure to approximate the ATLAS limit setting procedure. ATLAS use both 
CL S and profile likelihood methods to obtain a 95% confidence limit, using a full 
knowledge of the systematic errors on signal and background. Although the system- 
atic error on the background is provided in the ATLAS paper, we do not have full 
knowledge of the systematics on the signal expectation, which may in general vary 
from point to point in the Mq-M 1 / 2 plane. Rather than implement these statistical 
techniques, we take a similar approach to that used in |132| , and use the published 
a x A x e limits to determine whether a given model point is excluded in a search 
channel. We use our simulation to obtain the a x A x e value for a given model 
point, and consider the model to be excluded if the value lies above the limit given 
in table [4| This allows us to draw an exclusion contour in each search channel, 
and we estimate the combined limit by taking the union of the individual exclusion 
contours for each channel (i.e. the most stringent search channel for a given model 
is used to determine whether it is excluded). 

The procedure defined above neglects systematic errors on the signal and back- 



ground yields and, as noted in 132 [, this leads to a discrepancy between the Delphes 



results and the ATLAS limits in each channel. We follow 132 in using a channel 



dependent scaling to tune the Delphes output so that the limits in each channel 
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match as closely as possible "by eye". We obtain factors of 0.82, 0.85, 1.25, 1.0 and 
0.70 for the Rl, R2, R3, R4 and RHM regions respectively. Comparisons between 
the resulting Delphes exclusion limit and the ATLAS limit are shown in figure [2j 
where we observe generally good agreement in all channels. The largest discrepancy 
is observed in the RHM channel, where we find that one cannot get the tail of the 
limit at large Mq to agree with the ATLAS limit whilst simultaneously guarantee- 
ing good agreement at low Mo. This is likely to be due to the fact that we have 
effectively assumed a flat systematic error over the Mq-M 1 / 2 plane, whereas the 
ATLAS results use a full calculation of the systematic errors for each signal point. 
It is important to notice however that the combined limit will be dominated by re- 
gions Rl and R2 at low Mq, and thus by choosing to tune the RHM results in order 
to reproduce the large Mq tail, one can ensure reasonable agreement of the com- 
bined limit over the entire range. Where disagreement remains, the Delphes limit 
is less stringent than the ATLAS limit, and hence using it gives us a conservative 
estimate of the ATLAS exclusion reach. 



5.2.2 The importance of A and tan/3 

The ATLAS results in table [4] demonstrate a small excess in the central value of the 
observed yield in the high multiplicity channel, RHM. Although one should assert 
that this has an innocent explanation (mostly likely an underestimate of the number 
of high multiplicity events in the SM due to a deficient Monte Carlo generator), it 
provides motivation to consider SUSY models in which there is a smaller amount of 
coloured sparticle production than in the bulk of the low mass CMSSM parameter 
space. 

Such model points exist in the CMSSM at high- Mo and high-|Ao|, in which 
most of the squarks are heavy except one stop quark whose mass gets pushed to 
lower values due to a large splitting between the t\ and t 2 masses. These models 
furthermore exhibit low fine tuning, and would be capable of generating slightly 
higher masses for the lightest SUSY Higgs particle. The mass spectrum of one 
such point is shown in ngure[3l with M = 1440 GeV, M 1/2 = 177 GeV, tan/3 = 27, 
A = -2950 GeV and [i > 00 As ATLAS and CMS tighten the exclusion of SUSY 
models with several light squarks, models such as these are becoming much more 
important in the search for weak scale supersymmetry, and we therefore consider 
it important to add the effects of Ao and tan/3 to our handling of LHC SUSY 
constraints. 

The dependence on tan /3 is much weaker than that on Aq, as the ratio of Higgs 
doublet VEVs has a much greater impact on the Higgs sector of the CMSSM than 
on squark masses. However, large tan/3 values can reduce the stop and sbottom 
splitting induced by large values of Aq as mentioned in the previous paragraph, 
and potentially swap the mass ordering of the t\ and g with corresponding effects 
on the phenomenology. As inclusion of all four continuous CMSSM parameters 
is technically possible, and tan /3 may influence the phenomenology of zero-lepton 
channels in certain regions of the {Mo, M1/2? ^0} parameter space, we hence include 



it in this study. We assess the value of having gone to this effort in section |5.2.4 
once the method itself has been described. 



20 The point was found during a wide ranging scan of the CMSSM parameter space, hence the esoteric 
choice of parameters. 
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Figure 2: Comparison between Delphes and ATLAS 95% exclusion limits in the Mq-Mi/ 2 
plane, for the signal regions Rl, R2, R3, R4 and RHM defined in table[3j In the combined 
limit plot, the ATLAS limit is obtained using the ATLAS statistical combination, whilst 
the Delphes limit is obtained by taking the union of the Delphes limits for each signal 
region. 
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Figure 3: The sparticle mass spectrum for a point with large \A \ capable of generating 
a small excess of high multiplicity events at the LHC. For details, see main text. 



5.2.3 Fast simulation using machine learning techniques 

Running the entire chain of ISAJET, Herwig++, Delphes and PROSPINO for a given 
model point takes ~ 1 hour in total on a typical CPU. Although one can trivially 
parallelise the simulation of different model points, it is still infeasible to simulate 
all of the 2 x 10 6 posterior samples required to reweight an existing data set, let 
alone the 10 7 or more required if this was to be incorporated into the primary 
MultiNest run sequence for a full scan. If one were reweighting points after a scan 
had been completed, one could restrict the simulation to points that are reasonably 
probable, but even this still requires a very large number of CPU hours. It is this 
restriction that has prevented previous studies from considering the effects of tan /3 
and Aq. 

An obvious solution is to try and interpolate between a smaller grid of simu- 
lated values, such that one obtains a function that can give the signal expectation 
for any CMSSM point within fractions of a second. This is a standard regression 
problem, and there is an extensive collection of efficient techniques in the litera- 
ture for performing the interpolation. White, Buckley and Shilton have previously 



demonstrated good results in the CMSSM using machine learning algorithms 137 



including both a Bayesian Neural Network (BNN) and a Support Vector Machine 
(SVM), with a zero lepton signal region from the ATLAS 2010 analysis as the test 
case. Here we go much further by interpolating all of the ATLAS zero lepton search 
channel results (from the 2011 analysis) and by combining the search channels to 
reproduce the ATLAS combined exclusion result. 

Combining the search channels is non-trivial since ATLAS have not published 
enough information to determine the correlations between channels. We therefore 
continue to perform the approximate procedure outlined above. For any given 
model point, we can determine if it is excluded or still viable by choosing the 
most stringent limit on a x A x e for that point. Whilst this unfortunately only 
allows us to attach a discrete LHC-based likelihood to the points in the above 
posterior distributions, it is the most rigorous procedure that can be applied in the 
circumstances. We expect that this will slightly lower the apparent damaged done 
to the CMSSM by the ATLAS limits, as measured by the associated Bayes factor, 
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from what one would obtain with the full 4D likelihood. This is because we are 
effectively adding a significant amount of extra likelihood to all points which are 
"not excluded" (particularly those which are close to the limit), while removing 
likelihood from all "excluded" points. We expect the procedure to be adding more 
likelihood overall than is lost since points near the 95% confidence limit have quite 
low likelihood to begin with. Since it is an integral over the likelihood function 
which leads to the evidence values used in the Bayes factors, an overall increase in 
likelihood will increase the CMSSM evidence and thus lower the apparent damage 
to the CMSSM. Furthermore, we expect the 'true' likelihood map to quite sharply 
transition from strong to very weak exclusion of model points in the vicinity of the 
limit; for example the approximate 2D likelihood map computed in [27] shows this 
transition occurring over a range of around 50 GeV in M 1 / 2 . We thus expect any 
errors introduced into our analysis due to the step-function approximation to the 
limits and approximate combination procedure to be very small. 

Our study in |137| demonstrated successful interpolation of the signal expecta- 
tion itself. Given that we here want to apply only a discrete likelihood based on 
whether a point is excluded or not excluded, one can use a Bayesian neural net 
(BNN) as a classifier rather than a regressor (the former being the discrete case of 
the latter). For each channel in table [I] we have used the BNN implementation in 
the TMVA package |138 , 139| to classify SUSY parameter points into two classes: 



1. Excluded: (a x A x e x /) > / 

2. Not excluded: (a x A x e x /) < I 

where / is the limit for that channel given in table [4] and / is the scaling factor 
applied to the channel to obtain a close match with the ATLAS results. The success 
of the classification depends critically on the quality of the training data, and it 
is particularly essential to ensure that the training data adequately cover the limit 
(a x A x e x /) = I. In the Mq-M 1 / 2 plane, this limit is traced by the exclusion 
limits in figure [2| To maximise the accuracy of the BNN training in the region 
of maximum analysis sensitivity, while still achieving sufficiently comprehensive 
coverage of the M$-Mi/ 2 plane, we hence sample training data using a hybrid 
distribution composed of distinct two functions in Mq-Mi/ 2 : 

• uniform sampling in Mq E [10, 4000] GeV, and a falling exponential distribu- 
tion with width 500 GeV for M 1/2 E [10, 1000] GeV; 

• sampling from a ellipse with Gaussian profile, constructed such that it inter- 
sects the Mq axis at 1 TeV with width 300 GeV, and intersects the M 1 / 2 axis 
at 350 GeV with width 105 GeV. 

Sampling weight was distributed equally between these two distribution compo- 
nents, the resulting sampling density being shown in figure |4| Aq and tan f3 were 
sampled uniformly from Aq E [—3000, 4000] GeV and tan/3 E [0,62] regardless of 
the distribution type being used in Mq-M 1 / 2 . 

We generated two sets of training data of 25,000 points for the fi > branch, 
and a further 5,000 points on which to validate the classification performance. 

The output of the BNN classification is a mapping between the CMSSM input 
parameters Mo, M^, A) 3 tan/3, sign(/x) and a continuous variable that offers good 
discrimination between the "excluded" and "not excluded" points. Sample distri- 
butions of this variable (the "MLP Response" ) for "excluded" and "not excluded" 
points are shown in figure [5] for the ATLAS Rl search channel. By choosing a suit- 
able cut on this value, one can determine whether a given point is excluded given 
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Figure 4: The sampled distribution in the Mq~M 1 / 2 plane for BNN training, shown after 
removal of failed ISAJET and PR0SPIM0 points. 

the input parameters. The cut value must be chosen to provide a familiar compro- 
mise between efficiency and purity. A cut that is too low will lead to large numbers 
of points that are "not excluded" being classified as "excluded" . On the contrary, a 
cut that is too high will lead to large numbers of points that are "excluded" being 
classified as "not excluded" . We select our cut to minimise the former outcome- it 
is much worse to claim points are excluded when they are not excluded than to miss 
excluded points that should be excluded, since in the latter case one can present 
conservative results that, nevertheless, are not false. 

Figure [6] shows an example of this optimisation for the ATLAS Rl search chan- 
nel. The black line shows the fraction of SUSY points in our test sample of 5,000 
points that are labelled "excluded" when they should be "not excluded" vs the cut 
value on the MLP response. The red line shows the fraction of SUSY points that are 
labelled "not excluded" when the should be labelled "excluded"^} By choosing an 
MLP cut of 0.5, one can keep the fake exclusion rate below 5% whilst only missing 
10% of points which should be excluded. This is a very good performance consider- 
ing that we now have the ability to apply results to the full parameter space of the 
CMSSM. A summary of the performance for each channel after choosing suitable 
MLP response cut values is provided in table [5j There is an element of subjectivity 
in choosing suitable cut values. We do not allow the efficiency for excluding points 
to drop below 90%, but for channels where one can obtain a higher efficiency whilst 
keep the false exclusion rate below ~ 4% we choose the cut appropriately. 

Table [5] demonstrates that the false exclusion rate remains at the few per cent 
level in each search channel whilst we can exclude 90% of the points that should 
be excluded. We have succeeded in obtaining an efficient and robust classifier for 
SUSY model points. For the "ATLAS-sparticle" and "ATLAS-Higgs" data sets this 
classifier was incorporated into the full MultiNest run sequence, and thus used to 
concentrate the scans on regions considered "not excluded" by the classifier. 

21 All other points in the test sample are "not excluded" and labelled as such, or "excluded" and 
labelled as such. 
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MLPBNN response 

Figure 5: Distributions of the BNN response for "not excluded" points and "excluded" 
points, for the ATLAS Rl search channel. The MLPBNN response variable offers excel- 
lent discrimination between the two classes of SUSY model. 




MLPcut 

Figure 6: Fake exclusion rate and missed exclusion rate for the ATLAS Rl search channel 
vs the cut on the MLPBNN response. 



Region 


Rl 


R2 


R3 


R4 


RHM 


MLPBNN response cut value 
Fraction labelled "Excluded" when "Not Excluded" 
Fraction labelled "Not Excluded" when "Excluded" 


0.53 
3.2% 
10.0% 


0.51 
3.5% 
10.0% 


0.2 

3.2% 
6.8% 


0.45 
4.2% 
10.0% 


0.46 
3.5% 
8.1% 



Table 5: MLPBNN response cut values for each ATLAS search channel, with performance 
statistics for the chosen cut value. We choose to accept a lower rate of labelling excluded 
points as "not excluded" (and thus missing excluded points) to keep the rate of false 
exclusion low. 
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Figure 7: ATLAS 1 fb _1 sparticle search 95% confidence limits as estimated by the BNN 
classifier, displayed in the (M ,Mi/ 2 ) plane for various values of tan/3 (varying between 
plots) and Aq (curves as specified in the legend). The official ATLAS limit (red dashed), 
determined for Aq = and tan/3 = 10, is also displayed for comparison. 



5.2.4 Variation of exclusion limits with A and tan/3 

With the classifier trained we now reassess how worthwhile it was to estimate the 
full 4D limit. To do this we examine the position of the limit, as estimated by the 
classifier, in the (Mo, Mi/ 2 ) plane for a range of Aq and tan/3 values and compare 
these to the official ATLAS limit. A representative set of these limits is shown in 
figure [7| 

The classifier limit is observed to be largely unchanged from the ATLAS limit 
for Aq values between —2 and 3 TeV, for all tan/3 (with variations the most pro- 
nounced at low Mo), however for Aq outside this range the classier limit is seen to 
weaken in M 1 / 2l quickly dropping by around 100-200 GeV for Mq below about 1.5 
TeV, although structure exists in the limit allowing some points to remain excluded 
at higher M 1 / 2 . The size and details of this drop exhibit moderate tan/3 depen- 
dence. From this we conclude that Aq and tan/3 are certainly of importance to 
interpretations of LHC limits if regions with large Aq are of interest, but otherwise 
may be fairly safely neglected. 

We pre-empt our results to say that we find that there is not much posterior 
probability located outside —2 TeV < Ao < 3 TeV in any of our scans when using 
a log prior, and so our 4D treatment of the limit will have had little impact on 
our log prior results. However, when using the 'natural' prior we indeed find a 
significant amount of probability below Aq = — 2 TeV in all scans save the baseline 
scan, part of which would have been excluded had we not allowed the limit to vary 
with Aq. These posteriors are a by-product of our central results but we include 
them in Appendix [A] in part to illustrate this point. Further evidence of the impact 



of using a 4D limit can be seen in the profile likelihoods of figure [TOj from which 
it is clear that the lower limits at high \Aq\ allow extra points at very low Mq to 
escape exclusion. 

We next compare our findings on the Ao-tan/3 dependence of the sparticle search 
limits to those of other groups. In ref. \27\ more recent 4.7 fb _1 ATLAS and CMS 



limits 140, 141| were studied and no ylo-tan/3 dependence was observed within 



systematic uncertainties. To support this assertion the signal yield for a handful of 
points is presented, and shown to remain within systematics, however these all have 
Aq equal to either or 1 TeV, and our study agrees that little Ao-tan/3 variation 
should be seen in this range. The total Aq range scanned exceeded |5| TeV, so 
according to our findings some dependence may be expected, however since these 
are different limits to those we have used (as they were not yet available at the 



29 



time our computations were performed), it is plausible that their dependence on 
Aq is indeed weaker. This may occur because the 1 fb _1 ATLAS search we study 
contains a modest excess, which increases the likelihood for high Aq points, and 
thus increases the Ao dependence of the limit. In the 4.7 fb _1 search no excess as 
large as this was observed, so this extra source of Ao dependence may be absent. 

Older studies exist which also contribute to this picture. In ref. [28] a 35 pb _1 
limit was studied and it was concluded that assuming it to be Ao and tan/3 inde- 
pendent was a reasonable approximation, however it was also observed that points 
with high \Aq\ exhibited the largest disagreements with the Ao = limit, as we ob- 
serve. Furthermore, only one point outside —1.5 TeV < Ao < 2.5 TeV was studied 
(and this excluded for other reasons), well within the range our results indicate to 
be 'safe'. 

To conclude, the overall impact on our study of using the 4D limit appears 
to be minimal, however as limits increase to higher Mo and Mi/ 2 and larger \Aq\ 
values become more plausible (driven by a need to fit the 125 GeV Higgs candidate 
discussed in the next section) then it will become more important to use the 4D 
limit, with this importance increasing with the size of any excesses. The value 
of including Ao and tan/3 dependence in approximations to LHC search likelihood 
functions will thus need to be reassessed for each new limit which is produced. 



5.3 February 2012 ATLAS Higgs search results 

In December 2011 ATLAS and CMS released preliminary results for their com- 
bined Higgs searches |142 , 143| showing strong hints for the presence of a SM-like 
Higgs boson in the vicinity of 125 GeV. The official combinations were released in 



February 2012 75 , 144] with little change. Others have considered the impact that 



the existence of such a Higgs, if confirmed, would have on the CMSSM parameter 
space [26,27,33,39, 145 147] . We are interested in the current state of knowledge, so 
we do not make such assumptions. Instead, we reconstruct the full likelihood based 
on the public results. Our method bears some similarity to that used by |148| , 
however we work from signal best fit plots, not CLs limit plots. Since CMS do not 
produce signal best fit plots for all the channels we require we use only the ATLAS 
results. We will now detail our method. 

To construct signal best fit plots ATLAS and CMS use the log-likelihood ratio 
test statistic 



Q = -21og(A) = -21og 



£ 5 +&(A) 



-2 log 



/ P(data|m/ l , /i) 
\P(data|m/ l , ft) 



(26) 



Here rn^ is the Higgs mass parameter, \i the cross section scaling parameter (the 
factor which multiplies the SM prediction for the Higgs cross section for a given 
channel to achieve the hypothesised value, i.e. fi = ct/ctsm) and jl the value of \i 
which maximises the likelihood for a fixed value. Nuisance variables are profiled 
over. A zbla error band is also presented, the extents of which give the values of \i 
for which Q rises to 1 for each value of m^. Examples of such plots are shown in 



figure 8 of ref. 142 
Following ref. 



108 , if one assumes Wald's asymptotic approximation to be 



valid (which ATLAS confirms to be true to good accuracy for the three individual 
channels we use 72-74] as well as for the combination in ref. |75 ) then Q can be 
written as 

(/* - a) 2 



Q 



(27) 
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where it is assumed that jl is normally distributed with mean 11 and standard 
deviation a (when the data is generated by the signal plus background model with 
the parameters rrih and /i), and where both jl and a depend on the model parameters 
nrih and 11 we are testing. All the information regarding systematic and statistical 
uncertainties is carried by a. If a did not vary with 11 then we could immediately 
determine Q from the best fit and ±la curves (taking the largest deviation from \i as 
a to be conservative) of the published best-fit plots, and thus extract the likelihood 
ratio for all \± in each rrih slice. In fact this is exactly what we do, and this is safe 
because signals are at this stage small, which implies that the distributions of Q 
cannot be very different between 11 — jl and fi = (or else the establishment or 
exclusion of a signal at much higher significance would be possible). So assuming 
a to remain constant for each rrih is sufficient for our purposes. The reconstructed 
likelihood will be accurate near /i = /i and lose accuracy far from the best fit point, 
however the likelihood is low for such parameters so this mistake will have little 
impact on our results. 

We now can obtain the likelihood ratio for every value of 11 in each rrih slice, 
however the slices are not scaled correctly relative to each other. We can fix this 
by noting that the points 11 = for each rrih are degenerate (because rrih makes 
no difference to predictions if \i = 0). We can thus scale the likelihood of each rrih 
slice relative to the likelihood at ji = 0, i.e. instead of Q we can work with the test 
statistic Qcl s (so called because of its use in constructing CL S limits): 

,28) 

Applying Wald's approximation again we obtain [l08] 

/ P(data|m/,,^) ^ | / P(data|m /l , ii = 0) \ _ (ji - jl) 2 [i 



Qcl s °^ yP(data|m/ l , /i) J ~^ °^ ^ P(data|m/ l , jl) J a 2 a 2 

(29) 

As above, /i and a can be extracted for every rrih slice from the publicly available 
plots, but now the new term correctly normalises the slices relative to each other. 
The likelihood we extract contains an extra constant factor due to the \i — 
contribution however this is of no importance for our analysis. 

In figure [8] we show the likelihood function reconstructed from the ATLAS dipho- 
ton channel results, as an example. We checked the consistency of our reconstruc- 
tion by combining the three search channels we use and comparing the result to a 
reconstruction of the official ATLAS channel combination, finding good agreement. 
In our scans this likelihood is further convolved with a 1 GeV width Gaussian un- 
certainty in the rrih direction to account for theoretical uncertainty in the rrih value 
computed at each model point. 



6 Results 

6.1 Profile likelihoods and marginalised posteriors 

Before presenting our main results (the Bayes factors) we show ancillary results 
from the datasets that we used to calculate them. These are the profile likelihood 
functions and marginalised posterior PDFs over the CMSSM parameter space for 
each dataset, and may be found in figures 10][T3 in appendix [Aj These figures show 



the evolution of the profile likelihoods and posteriors from the "pre-LEP" situa- 
tion (first row), to including the LEP and the XENON100 data (second row), to 



31 



b 3 



1 






i 


: 






: 


: 






: - 












A 






, A 




A 


A 



115 120 125 130 135 140 145 
m h (GeV) 




115 120 125 130 135 140 145 
m h (GeV) 



Figure 8: Reconstructed likelihood function for the diphoton channel, using asymptotic 
approximations for the test statistic distributions. In the left frame a reproduction of the 
ATLAS signal best fit plot from ref. [72] (with ±la band) is shown, while on the right 
is the reconstructed A% 2 map, with the A% 2 = 1,4,9 contours shown. We ignore the 
negative <j/<Jsm region as it is not relevant for the models we consider. In our scans this 
likelihood (and those for the other channels) is convolved with a further 1 GeV Gaussian 
on rrih to account for theoretical/numerical uncertainty in the value of rrih computed at 
each model point, and so the best fit region is extended in rrih by an extra GeV or so. 



adding the LHC sparticle searches (third row), to folding in the 2012 February 
Higgs search results. The figures reflect the well known effect: LEP has pushed the 
viable sparticle masses upwards substantially. Specifically, LEP eliminated some 
of the lowest M\/2 region, the region with the lowest fine-tuning, and created the 
small hierarchy problem. The LHC sparticle searches directly lower the likelihood 
only in the lowest Mq-Mi/ 2 corner. This leaves the bulk of the highest likelihood 



region toward slightly higher Mo and M^ 22 The 2012 February Higgs data seri- 
ously damages the high likelihood region at the lowest Mo-M^, resulting in the 
relative enhancement of high negative Ao regions with high Higgs masses, and the 
likelihood is pushed toward even higher M 1 / 2 - Interestingly the highest likelihood 
region hardly moves, despite predicting a Higgs mass much below 125 GeV (it is 
instead around 115 GeV). This is because the ATLAS Higgs signal is not yet strong 
enough to conclusively outweigh the observables which strongly favour the low mass 
region, particularly 5a M , however extremely strong tension is created which causes 
the evidence to drop significantly and the Bayes factor to strongly disfavour the 



CMSSM. As can be seen in the profile likelihoods of figure [TT] and the Bayes fac- 
tors of figure [9j removing the Sa^ constraint indeed goes a significant way towards 
relieving this tension and reducing the total damage to the CMSSM. At the same 
time the mid-tan /3 region emerges with the highest likelihood. 

The evolution of the marginalised posteriors follows a similar pattern. The 68 
and 95 percent credibility regions follow the general trend of the highest likelihood, 
moving toward higher Mq and A notable difference between the log and 

natural prior cases is that in the natural prior case the posterior exhibits a strong 
preference for tan/3 < 10, where the \i fine tuning is generally low, which is de- 
creased only a small amount by the new data, while in the log prior case there is 
clear movement of the preferred regions to higher tan /3. In conjunction, in order 
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The apparent 'thinning' of the likelihood toward higher Mq and Mi/ 2 is a mere sampling artefact. 
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to maintain low tan/3, the natural prior scan is forced towards large negative Aq 
values, while the log prior viable regions end up centred on = 0, although with 
sizable variance. 



6.2 Bayes factors and their interpretation 

Based on the likelihood functions and posterior probabilities shown in the previous 
section, we have computed the Bayes factors which update the odds of the CMSSM 
'existing' relative to our SM-like reference model for several data changes. As 



described in section 3.2 the following data sets were utilised in our study: 



Pre-LEP: All constraints listed in tableware imposed except for the LEP Higgs 
lower bound, the XENON100 limit, the ATLAS direct sparticle search limits, 
and the ATLAS Higgs search results. 

LEP+XENON100: As Pre-LEP, but including the LEP Higgs and XENON100 
limits. 

ATLAS-sparticle: As LEP+XENON100, but including the ATLAS direct spar- 
ticle search limits. 

ATLAS-Higgs: As ATLAS-sparticle, but including the ATLAS Higgs search re- 
sults. 

The evidence for each dataset is computed in the \i > branch of the CMSSM, 



for both the log and natural prior as described in section |4.1| giving us a total of 
8 data sets which have resulted from around 100 million likelihood evaluations in 
total. From these evidences we compute the Bayes factors for the data transitions 

Pre-LEP -> LEP+XENON100 
LEP+XENON100 ATLAS-sparticle 
ATLAS-sparticle ATLAS-Higgs 



according to the prescription of eq. (10) and (12), for each choice of prior. We also 
compute a 'cumulative' Bayes factor by multiplying together the individual Bayes 
factors in the sequence of transitions. We present the results in table [6] and figure 

m 

The first column of table [6] lists the datasets we have computed. The second 
shows the log evidence value \nZ and its statistical uncertainty as computed by 
MultiNest, using the method described in section [4j except in the case of the 



SM evidences, where A In Z values are computed as described in section |3.3| In 
the fifth column the Bayes factor B is shown for the transition to the dataset of 
each row from that of the previous row, followed by the cumulative Bayes factor 
^cumulative > which is the product of B with all of the previous Bayes factors, in 
column six. Columns three and four show the breakdown of each Bayes factor into 
the maximum likelihood ratio for the new data and the respective Occam factors 



for each model, as defined in eq. (11). The final column offers an interpretation of 
the strength of each Bayes factor according to the Jeffreys scale as listed in table [HJ 
A graphical representation of these results (and those of table [7| is presented in 
figure |9| 

Table [6] and figure [9] show that the LEP Higgs limit very strongly reduced our 
trust in the low mass CMSSM. The LHC sparticle limits induced a much smaller 
and not very significant additional reduction, and finally the LHC Higgs signal 
hints cause a 'substantial' additional swing against the CMSSM. The combined 
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Scenario 


In 2 


LR 


O 


B 


-^cumulative 


Strength of B 


SM 


Pre-LEP 


0* 












LEP+XENON100 


-1.26 




1 : 3.52 








ATLAS-sparticle 


-1.26 




1 : 1 








ATLAS-Higgs 


-5.16 




1 : 33.3 








CMSSM (log priors) 


Pre-LEP 


54.30(2) 












LEP+XENON100 


50.34(2) 


1 : 1 


1 : 51.9(1) 


1 : 14.7(4) 


1 : 14.7(4) 


Strong 


ATLAS-sparticle 


49.62(2) 


1 : 1 


1 : 2.04(5) 


1 : 2.04(5) 


1 : 30.1(8) 


Barely worth mentioning 


ATLAS-Higgs 


43.91(2) 


1 : 1.8 


1 : 113(3) 


1 : 6.1(2) 


1 : 185(5) 


Substantial 


CMSSM (natural priors) 


Pre-LEP 


44.73(2) 












LEP+XENON100 


40.54(2) 


1 : 1 


1 : 65.6(2) 


1 : 18.6(6) 


1 : 18.6(6) 


Strong 


ATLAS-sparticle 


39.87(2) 


1 : 1 


1 : 1.97(6) 


1 : 1.97(6) 


1 : 37(1) 


Barely worth mentioning 


ATLAS-Higgs 


34.29(2) 


1 : 1.8 


1 : 102(3) 


1 : 5.4(2) 


1 : 197(6) 


Substantial 



We have computed AlnZ directly for the SM so this zero is an arbitrary initial value, for illustrative 
purposes only. 



Table 6: Summary and interpretation of our results. The global log evidence values 
InZ and statistical uncertainties are presented as computed by MultiNest for each scan, 
except in the case of the SM for which we have computed A In Z values directly. The 
Bayes factor B is shown for the transition to the data set of each row from that of the 
previous row. The cumulative Bayes factor £> cumu i ative is the product of B with all of 
the previous Bayes factors. The components of the Bayes factor are also shown: the LR 
column shows the maximum likelihood ratio between the SM and CMSSM for the newly 
added data (which is only different from one for the ATLAS Higgs search data, where we 
see that the maximum likelihood is a factor of 1.8 higher in the SM than the CMSSM), 
and the O column shows the Occam factors. The final column offers an interpretation of 
the strength of each Bayes factor according to the Jeffreys scale as listed in table |8| The 
combined effect of all experiments on a 'pre-LEP' odds ratio is seen to be a 'Decisive' 
shift away from the low energy CMSSM when judged by the Jeffreys scale, using either 
prior. 
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effect of all experiments (aside from the LHC Higgs data) on a pre-LEP odds ratio 
is seen to be a shift against the low mass CMSSM of a strength above the level 
considered 'Decisive' on the Jeffreys scale. These findings are robust against the 
shapes of the prior probabilities of the CMSSM parameters that we have considered, 
although they would be weakened by priors which strongly favoured high Mo and 
M1/2 values. Presently the impact of XENON100 is negligible, but we remind the 
reader that the apparent strength of each piece of data is dependent on the order in 
which it is added. The XENON100 results appear largely irrelevant because they 
exclude regions of the CMSSM parameter space already excluded by the LEP Higgs 
searches and have only a small impact on the surviving parameter space. One may 
expect that the reinforcement of previous exclusions by independent experiments 
should count for something in the Bayesian framework (i.e. as reassurance that 
no mistakes were made by either experiment), however in the current analysis all 
data in the likelihood function is assumed to be 100% reliable and so we are not 
considered to learn anything new by "doubling up". In order to see such effects 
in an analysis a measure of doubt about the reliability of experimental data would 
need to be introduced. 

It has been noted previously that the Sa^ constraint is in considerable tension 
with several other observables 1 58 , 59 and indeed this tension plays a strong role 
in the damage to the CMSSM that we observe since Sa^ strongly favours the now 
excluded lowest mass regions. However, there remains some controversy over its 
value [96l p9Hl53] , so we consider the impact on our inferences if we remove it 



from our likelihood function. It is too computationally expensive to do this by 
completing a full set of new scans, so we subtract it from the likelihood function 
of our original data sets in a similar 'afterburner' manner as is done in ref. [32] . 
The accuracy of the results obtained this way is lower than those obtained from 
full scans, and from investigations of obtaining our primary results by reweighting 
we expect the evidences obtained this way to overestimate the damage a factor of 



the order of 2 or 3 A factor of two change to the Bayes factor may sound like a 
very large mistake, however if we consider the Jeffreys scale a reliable assessment of 
Bayes factor strength we can note that its logarithmic nature renders factors of the 
order of 2 or 3 'barely worth mentioning'. We can thus be confident that the Jeffreys 
interpretations of the reweighted results are reliable despite the error -so long as 
we do not allow several errors of this size- so we present them in table [7| When a 
factor of 2 reduction in Bayes factor would downgrade the Jeffreys interpretation 
we have done so to be conservative. 

Since the 8a ^ constraint pushes the posteriors strongly down in Mq and M]y 2 , 
removing it makes us much less surprised that no direct evidence for the low-mass 
CMSSM was seen at LEP or in the LHC sparticle searches. This is reflected in the 
weaker Bayes factors than in table[6| The LEP results in particular are seen to cause 
much less damage. The combined effect of both colliders on a pre-LEP odds ratio 
is seen to be greatly reduced; for log and natural priors the final cumulative Bayes 
factors are weakened to 'Substantial' and 'Strong' shifts away from the CMSSM 
respectively, also demonstrating through the increased prior dependence that Sa^ 
plays an important role in constraining the initially viable parameter space, i.e. in 
building the informative priors from the "pre-LEP" dataset. 

As mentioned in section |4.1| we were driven to ignore the n < branch of the 
CMSSM parameter space by computational restrictions and due to its poorer fit to 
data, particularly 5a M . However, given the significant (relative) boost to confidence 
in the CMSSM that is gained by removing 5a M , and the potential for the boost to 
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Since the thorough scan finds many new good-fitting points that the reweighting neglects. 
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be even larger had the fi < branch not been ignored, it would be interesting to 
take this branch into account in future work. Confidence in the da^ constraint is 
thus seen to remain an important issue. 

We understand that some readers may remain confused as to how the removal of 
the Sdfj, constraint can improve the performance of the CMSSM in our analysis. To 
understand this, it is important to remember that the 8a^ constraint entered into 
our analysis as part of our 'baseline' dataset, which was used to effectively create 
an informative prior for the CMSSM (i.e. the posterior resulting from the inclusion 
of this baseline data). The only effect of 5a ^ is thus to help determine the initially 
viable regions of parameter space in each model. Since the SM is highly constrained 
it cannot 'tune' its prediction of Sa^ to match experiments, so there is no change to 
its parameter space whether Sa^ is included or not. On the other hand, the initially 
viable parameter space of the CMSSM is severely restricted -to low Mq and M 1 / 2 
values- by the demand that it reproduce the observed Sa^ value. This leaves the 
CMSSM highly vulnerable to damage from the LEP2 Higgs limits, which is reflected 
in the large Bayes factor against it when the LEP2 data is introduced. Removing 
Sa^ alleviates this extremely strong tension, greatly reducing the corresponding 
Bayes factor. The relative maximum likelihood penalty against the SM that one 
may expect to be present due to 5a M is not present in our Bayes factors, because we 
have separated it into the prior odds. Our analysis is not designed to assess these 
prior odds, however when interpreting our Bayes factors the reader must keep in 
mind which data we have dealt with in the Bayes factors and which they are left 
to consider in their personal prior odds. 

To reiterate: the reader may feel that we are not considering the direct effects 
of the various observables that form our initial "pre-LEP" data set on the model 
comparison. This is completely true; all this data has contributed to previous 
'iterations' of the Bayesian process, and must be considered in the prior odds for 
the current analysis. We have done this to distance our analysis as much as possible 
from the subjective initial parameter space priors, in order to most accurately 
isolate the impact of the experiments under study. From the robustness of the 
results presented in tables [6] this appears to have been achieved, although some 
prior dependence remains if the Sa^ constraint is removed, as the results of table 
[7] show. The cost of this robustness is that some of the burden of interpretation 
remains on the reader. For example, say at the "pre-LEP" stage one believes the 
odds for the CMSSM vs the SM-like model to be 1:1, after considering all the data 
in our 'baseline' set along with personal theoretical biases. Our Bayes factors then 
dictate only how one is required to modify these beliefs in light of the more recent 
data. As a more explicit demonstration of how this should be done we offer the 
following toy thought process: 

(First considering the baseline data): "The Standard Model does not predict 
Sa^ correctly, it does not provide any insight into the dark matter problem, and I 
consider the hierarchy problem to be a serious issue; however, the Standard Model 
has otherwise proven itself a simple and reliable model. On the other hand, the 
CMSSM can simultaneously predict Sa^ correctly, provide a dark matter candidate, 
and largely solve the hierarchy problem (remembering to ignore LEP2 and LHC 
information for now). However, it makes some powerful assumptions regarding the 
mechanism by which supersymmetry is broken that I do not find appealing. Based 
only on this information I would be prepared to make a bet at 50:1 odds that the 
CMSSM (with M and M 1/2 below 2 TeV) is the more accurate model of Nature" 
(a very strong belief as measured by the Jeffreys scale; the toy thinker is quite a 
'hardcore' CMSSM aficionado). 
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(Next considering the Bayes factors of table [6]): "According to Balazs et al. the 
total CMSSM:SM Bayes factor for learning the LEP2 Higgs limits, XenonlOO limits, 
1 fb _1 sparticle search limits, and early 2012 Higgs search results, is about 1:200. 
The parameter space priors they have used roughly correspond to my expectations 
about the CMSSM, so I accept this number. Multiplying this by my prior odds 
I obtain posterior odds of about 1:4, now in favour of the Standard Model by a 
moderate amount. I thus expect to lose my initial bet, and decide to focus my 
research efforts on models besides the low-mass CMSSM" 

Although crude, and not rigorous as to the details of settling bets, we hope that 
this example helps to clarify the meaning of our results. 

While this paper was under review the ATLAS and CMS Higgs searches made 
significant progress, with the local p-values for the signal "hints" utilised in this 
work increasing in significance to over "5 sigma", leading to the announcement of 
the discovery of a new integer-spin resonance |154 , 155] . We expect this to increase 
the degree to which the CMSSM is disfavoured in our results, since the decrease 
in parameter space compatible with the stronger measurement will be larger in the 
CMSSM than the SM, and potential discrepancies in the branching ratios from SM 
predictions are not significant enough to make much of an impact. In addition, 
ATLAS and CMS have released the res ults of nu merous supersymmetry searches 
using up to 5 fb _1 of data (for example |l40[|l41| , utilising the full 2011 data set; 
results of similar strength utilising 2012 data also exist, but no combination of 2011 
and 2012 data is yet available), a significant increase over the 1 fb _1 results we have 
used. No hints of new physics have been seen, and though the improved limits do 
cut a small way into the posterior remaining in our final "ATLAS-Higgs" datasets 
the improvement is not sufficient to significantly alter the Bayes factors we have 
computed; we expect considerably less than an extra factor of two shift against the 

cmssmEB 



7 Conclusions 

We examined the viability of the low energy CMSSM, the corner of the parameter 
space with Mq and M 1 / 2 restricted below 2 TeV, in the light of data from before 
LEP to the recent measurements of the LHC. To quantify this viability we computed 
the Bayes factors associated with learning the LEP Higgs limits, XENON100 dark 
matter limits, LHC sparticle searches, and the 2012 LHC Higgs hint, in sequence, in 
a straightforward Bayesian hypothesis test of the CMSSM against a SM-like model. 

Interpreting the relative change of belief in the CMSSM induced by these Bayes 
factors in terms of the Jeffreys scale we concluded the following. The LEP Higgs 
limit strongly reduced our trust in the low energy CMSSM, as is well known. The 
LHC sparticle limits deal a much smaller and not yet very significant additional 
blow. Lastly, the LHC Higgs hints are already strong enough that they have a 
substantial impact even if the previous damage is ignored. When considering the 
cumulative effect of all three data changes we found that support for the CMSSM, as 
measured by the posterior odds, is reduced relative to the SM-like alternative by a 
decisive 200 fold. These findings are robust against the shape of prior probabilities 
of the CMSSM parameters we considered (and are expected to remain so under 
other reasonable choices for priors), however they are severely weakened if the 



4 We note that the new limits cut off much less than half of the posterior remaining in the "ATLAS- 



Higgs" dataset (shown in the last frame of figure 13), so the corresponding Bayes factor is likewise much 
less than two. 
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Scenario In Z 


LR 


O 


B 


-^cumulative 


Strength of B 


CMSSM (log priors) 


Pre-LEP 


36.69(2) 












LEP+XENON100 


34.43(2) 


1 : 1 


1 : 9.6(2) 


1 : 2.72(6) 


1 : 2.72(6) 


Barely worth mentioning 


ATLAS-sparticle 


34.77(2) 


1 : 1 


1 : 0.72(2) 


1 : 0.72(2) 


1 : 1.95(5) 


Barely worth mentioning* 


ATLAS-Higgs 


29.42(2) 


1 : 1.8 


1 : 78(2) 


1 : 4.2(2) 


1 : 8.3(1) 


Barely worth mentioning 


CMSSM (natural priors) 


Pre-LEP 


29.29(2) 












LEP+XENON100 


27.27(2) 


1 : 1 


1 : 7.6(2) 


1 : 2.15(6) 


1 : 2.15(6) 


Barely worth mentioning 


ATLAS-sparticle 


26.67(2) 


1 : 1 


1 : 1.81(6) 


1 : 1.81(6) 


1 : 3.9(1) 


Barely worth mentioning 


ATLAS-Higgs 


20.87(2) 


1 : 1.8 


1 : 126(4) 


1 : 6.7(2) 


1 : 26.1(8) 


Substantial 



This Bayes factor appears to indicate a slight increase in the viable CMSSM parameter space, which is 
impossible. It is therefore certain to be an artefact of reweighting process. 



Table 7: Summary and interpretation of our results, with the Sa^ constraint removed. 
Columns as in table |6| We have dropped the SM rows because they are unchanged from 
table |6j The Sa^ constraint pushes the posteriors strongly down in the mass parameters, 
so removing it makes us much less surprised that no direct evidence for the low-mass 
CMSSM was seen at LEP or in the LHC sparticle searches. This is reflected in the 
weaker Bayes factors than in table [6j The LEP results in particular are seen to be much 
less surprising. The combined effect of both colliders on a 'pre-LEP' odds ratio is seen to 
be downgraded from a 'Decisive' to 'Substantial' (by the Jeffreys scale) shift away from 
the low energy CMSSM when using the log prior, and to be downgraded from 'Decisive' 
to 'Strong' when using the natural prior (where dividing the cumulative Bayes factors by 
2 to account for reweighting errors would not change these interpretations). Confidence 
in the Sa^ constraint is thus seen to have a very large impact, and is likely to remain an 
important issue in models beyond the CMSSM. 



B 


Strength of evidence 


< 1 : 1 
1:1 to 3:1 
3:1 to 10:1 
10:1 to 30:1 
30:1 to 100:1 
> 100 : 1 


Negative 

Barely worth mentioning 

Substantial 

Strong 

Very strong 

Decisive 



Table 8: The Jeffreys scale for interpreting Bayes factors. We use this scale to interpret 
our results for B and £> cumu i ative . 
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Figure 9: Bayes factors for the three data changes we consider (right), for a hypothesis 
test of the CMSSM against the Standard Model (SM), as computed using both 'log' and 
'CCR' priors for the CMSSM and both with and without the 8a^ constraint imposed. We 
begin by transitioning from the "pre-LEP" situation to including the LEP Higgs search 
and the XENON100 data (red), to adding the ATLAS 1 fb _1 sparticle searches (blue), 
to folding in the 2012 February ATLAS Higgs search results (green). We also show the 
breakdown of each Bayes factor (left) into the maximum likelihood ratio of the data 
added in each transition (yellow highlight), and the "Occam" factors for each transition 
for both the SM (blue highlight) and the CMSSM (remainder). If one was willing to bet 
even odds on the CMSSM and SM at the "pre-LEP" stage, the product of these Bayes 
factors (as stacked) give the posterior odds with which one should now gamble on these 
models, given our parameter space priors and data assumptions. The cumulative effect 
of these Bayes factors is an almost 200 fold swing in the odds away from the CMSSM, 
reduced to a 10-30 fold swing if the 8a /2 constraint is dropped. Bayes factors of the former 
strength represent significant experimental disfavouring of the low energy CMSSM and 
could only be outweighed by very strong prior odds (determined by considerations outside 
the scope of our analysis), while the latter values (with Sa^ removed) are of only moderate 
strength and are unlikely to dominate over prior considerations. Despite differences in 
the details of posteriors obtained under the two priors used, the Bayes factors themselves 
remain remarkably robust, although this robustness is partially compromised if 8a ^ is 
ignored since it is a powerful constraint which helps the baseline ( "pre-LEP" ) likelihood 
to dominate over differences in priors. 
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sometimes contentious muon anomalous magnetic moment constraint is removed 
from consideration. Presently the impact of XENON100 is negligible, although in 
the near future dark matter direct detection is expected to further reduce our belief 
in the low energy corner of the CMSSM, unless they discover a positive signal soon. 

The strength of these results is largely due to the very small amount of CMSSM 
parameter space in the posterior of the initial ( "pre-LEP" ) data set which is capable 
of producing a lightest Higgs of around 125 GeV, in accordance with the LHC Higgs 
hints, and so is quite expected from that perspective. The ease with which this can 
be accommodated in the SM-like model causes the more 'wasteful' CMSSM to be 
strongly disfavoured. The CMSSM would not fare as poorly in a test against a 
perhaps more realistic model of similar parameter space complexity, unless that 
model naturally produces a compatible Higgs in a much more substantial portion 
of its otherwise viable parameter space. Likewise if there exists a good reason to 
restrict the parameter space prior for the CMSSM to those regions that produce 
a relatively viable Higgs, such as some motivation from a higher energy theory, 
then our large penalising Occam factors may be largely negated. This is essentially 
the Bayesian manifestation of a naturalness problem; the CMSSM is now a highly 
unnatural model (completely separately from the little hierarchy problem, which 
is associated with data in our baseline set) due to the small amount of parameter 
space capable of fitting both the Higgs observations and previous data, and this 
is strong motivation to search for a more complete theory (if not for a completely 
different theory) to explain why this small portion of parameter space should be 
chosen by Nature. 
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A Plots of CMSSM profile likelihoods and 
marginalised posteriors 

This appendix contains the figures referred to in section |6j We refer the reader to 
that section for further information. 
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Figure 10: The evolution of the profile of the (log-) likelihood function from the "pre-LEP" 
situation (first row), to including the LEP Higgs search and XENON100 data (second 
row), to adding the 1 fb _1 LHC sparticle searches (third row), to folding in the 2012 
February Higgs search results. Contours containing 68% and 95% confidence regions are 
shown. The above results were obtained using the log prior. Results obtained using the 
CCR prior (not shown) show variations consistent with the different sampling density 
but are qualitatively similar. 
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Figure 11: The evolution of the profile of the (log-) likelihood function from the a pre-LEP" 
situation (first row), to including the LEP Higgs search and XENON100 data (second 
row), to adding the LHC sparticle searches (third row), to folding in the 2012 February 
Higgs search results. Contours containing 68% and 95% confidence regions are shown. 
The above results were obtained using the log prior and have been reweighted to estimate 
the effect of removing the 8a ^ constraint. Significant deterioration of the sampling is seen 
due to the shift in the preferred regions away from the originally sampled regions, however 
the general impact of removing the Sa M constraint can be seen in the motion of the 
preferred regions upwards in the mass parameters. Of particular note is the very strong 
shift to high A when the ATLAS Higgs search results are imposed, which is much less 
pronounced in figure [lOj indicating very strong tension between the ATLAS Higgs search 
results and the 5a /2 constraint. Results obtained using the CCR prior (not shown) show 
variations consistent with the different sampling density but are qualitatively similar. 
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Figure 12: The evolution of the CMSSM marginalised posterior probability distribu- 
tions from the "pre-LEP" situation (first row), to including the LEP Higgs search and 
XENON100 data (second row), to adding the LHC sparticle searches (third row), to fold- 
ing in the 2012 February Higgs search results. Log priors are used and 68% and 95% 
credibility regions are shown. 
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Figure 13: The evolution of the CMSSM marginalised posterior probability distribu- 
tions from the "pre-LEP" situation (first row), to including the LEP Higgs search and 
XENON100 data (second row), to adding the LHC sparticle searches (third row), to fold- 
ing in the 2012 February Higgs search results. Natural ("CCR") priors are used and 68% 
and 95% credibility regions are shown. The natural prior can be seen to favour lower M 
and tan /3 than the log prior. 
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